Second Life of a Hungarian SharePoint Geek

September 2, 2018

How to delete events from a SharePoint calendar without messing up the Recycle Bin

Filed under: Bugs, Calendar, SP 2013 — Tags: , , — Peter Holpar @ 09:01

Recently we detected that a lot of deleted calendar entries appeared in the Recycle Bin of a SharePoint site. They were all originated from a specific calendar instance, and deleted by a custom application (written in C#) that is scheduled to run regularly to purge old entries.

Although we could have deleted the items using the bulk method described here, in this case we deleted the entries one by one. To get IDs of the “master” entries (entries that are not recurring event exception) we should delete, we run a CAML query like the below one, filtering on the EventType field of the items, just as described in my former post:

<Where>
    <And>
    <Lt>
        <FieldRef Name=’EndDate’ />
        <Value Type=’DateTime’>2018-01-01 00:00:00</Value>
    </Lt>
    <Lt>
        <FieldRef Name=’EventType’/>
        <Value Type=’Integer’>2</Value>
    </Lt>
    </And>
</Where>

You can read more about the meaning of the various EventType values in this blog post.

As I wrote in my post, we should (at least, theoretically, see explanation later) delete only the main entries, as all of the related entries (the recurring event exceptions, deleted and changed instances of the series) are deleted automatically by the system when you delete the main entry.

Having the IDs from the CAML query, we deleted the entries by iterating through the collection of IDs and invoked the the DeleteItemById method of the SPListItemCollection method, like:

foreach (int itemIDToDelete in itemIDsToDelete)
{
    calendarList.Items.DeleteItemById(itemIDToDelete);
}

As you probably know, you can delete an item from code by invoking the Delete method of the SPListItem instance of the item (in this case the item is deleted immediately, without being recycled), or recycle it by calling the Recycle method of the SPListItem instance (in this case the item is simply moved to the Recycle Bin and you can restore it later if you wish). I should point out, that both of these methods call internally the private DeleteCore method of the SPListItem class, using the parameter value DeleteOp.Delete in the first case and DeleteOp.Recycle in the second case.

The DeleteItemById method invokes the Delete method however, so it should not miss up the Recycle Bin, as it obviously did in our case.

public void DeleteItemById(int id)
{
    this.GetItemById(id).Delete();
}

So what was the problem? After a short investigation and running a few tests, we found, that only recurring event exceptions got moved to the Recycle Bin when the system delete them automatically, the main entries were deleted permanently. It means, that the Delete method of the SPListItem class is buggy and the same is true for the DeleteItemById method, at least I don’t consider this behavior to be some kind of hidden feature.

How to solve the problem? If you have a lot of recurring event exceptions in your calendar, and don`t want to pollute your Recycle Bin, the best you can do to create some kind of extension method that deletes the related entries (recurring event exceptions) explicitly, not letting the system to delete them automatically.

I’ve created two extension methods, as shown below:

  1. public static void DeleteItemByIdIncludingRecurringEventExceptions(this SPListItemCollection items, int id)
  2. {
  3.     items.GetItemById(id).DeleteIncludingRecurringEventExceptions();
  4. }
  5.  
  6. public static void DeleteIncludingRecurringEventExceptions(this SPListItem item)
  7. {
  8.     if (item == null)
  9.     {
  10.         throw new ArgumentNullException("item");
  11.     }
  12.  
  13.     if (!item.ContentTypeId.IsChildOf(SPBuiltInContentTypeId.Event))
  14.     {
  15.         throw new ArgumentException(string.Format("Item must have a content type of Event ({0}) or a content type derived from that", SPBuiltInContentTypeId.Event), "item");
  16.     }
  17.  
  18.     // we need to perform the check only if the item is a main entry of a recurring event series
  19.     // in this case, EventType should be 1, see
  20.     // https://aspnetguru.wordpress.com/2007/06/01/understanding-the-sharepoint-calendar-and-how-to-export-it-to-ical-format/
  21.     var eventType = item["EventType"];
  22.     if ((eventType is int) && ((int)eventType == 1))
  23.     {
  24.         SPList list = item.ParentList;
  25.         int itemId = item.ID;
  26.  
  27.         // querying recurring event exceptions that belong to the current item
  28.         SPQuery query = new SPQuery();
  29.         SPListItemCollection itemsToDelete = null;
  30.         query.Query = String.Format(@"<Where><And><Gt><FieldRef Name='EventType'/><Value Type='Integer'>2</Value></Gt><Eq><FieldRef Name='MasterSeriesItemID'/><Value Type='Integer'>{0}</Value></Eq></And></Where>", itemId);
  31.         itemsToDelete = list.GetItems(query);
  32.  
  33.         //
  34.         List<int> itemIDsToDelete = itemsToDelete.Cast<SPListItem>().Select(i => i.ID).ToList();
  35.  
  36.         itemIDsToDelete.ForEach(i =>
  37.             {
  38.                 SPListItem subItem = list.GetItemById(i);
  39.                 try
  40.                 {
  41.                     subItem.Delete();
  42.                 }
  43.                 catch (Exception ex)
  44.                 {
  45.                     // error when deleting a recurring event exception
  46.                     // as a possible workaround, convert it to a standard event and try to delete it again
  47.                     subItem["EventType"] = 0;
  48.                     subItem.Update();
  49.                     subItem.Delete();
  50.                 }
  51.             });
  52.     }
  53.     // finally, delete the main entry as well
  54.     item.Delete();
  55. }

You can use the DeleteItemByIdIncludingRecurringEventExceptions method in place of the DeleteItemById method and DeleteIncludingRecurringEventExceptions method in place of the Delete method. We search for the related items by using the MasterSeriesItemID field value in the CAML query. You can use these methods only for items having the Event content type or a custom content type derived from it.

You can use the methods like this:

var list = web.Lists["Calendar"];          
list.Items.DeleteItemByIdIncludingRecurringEventExceptions(10);

Note, that we also had some corrupted recurring event exceptions in our calendar, probably created automatically by a faulty application. Although we could get a reference for the items itself (for example, by calling the GetItemById method), and change its properties if we wished, we got the exception below if we tried to delete them from code, or even if we only wanted to display the items from the All Events view in the browser.

Item does not exist. The page you selected contains an item that does not exist. It may have been deleted by another user.<nativehr>0x81020016</nativehr><nativestack></nativestack>
SPRequest.DeleteItem: UserPrincipalName=i:0).w|s-1-5-21-3634847118-2359816030-2114994487-3414, AppPrincipalName= ,bstrUrl=http://YourServer/Web/SubWeb ,bstrListName={A38F8D71-F481-4A93-85B8-AC42BB2BE6EC} ,lID=4596 ,dwDeleteOp=3 ,bUnRestrictedUpdateInProgress=False
System.Runtime.InteropServices.COMException: Item does not exist. The page you selected contains an item that does not exist. It may have been deleted by another user.<nativehr>0x81020016</nativehr><nativestack></nativestack>, StackTrace:    at Microsoft.SharePoint.SPListItem.DeleteCore(DeleteOp deleteOp)     at Microsoft.SharePoint.SPListItem.Delete()     at CallSite.Target(Closure , CallSite , Object , Int32 )     at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)     at System.Management.Automation.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.Ente…
…rTryCatchFinallyInstruction.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame)     at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0)     at System.Management.Automation.DlrScriptCommandProcessor.RunClause(Action`1 clause, Object dollarUnderbar, Object inputToProcess)     at System.Management.Automation.DlrScriptCommandProcessor.Complete()     at System.Management.Automation.CommandProcessorBase.DoComplete()     at System.Management.Automation.Internal.PipelineProcessor.DoCompleteCore(CommandProcessorBase commandRequestingUpstreamCommandsToStop)     at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input, Hashtable errorResults, Boolean enumerate)     at S…
…ystem.Management.Automation.Runspaces.LocalPipeline.InvokeHelper()     at System.Management.Automation.Runspaces.LocalPipeline.InvokeThreadProc()     at System.Management.Automation.Runspaces.PipelineThread.WorkerProc()     at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)     at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)     at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)     at System.Threading.ThreadHelper.ThreadStart()
 

The simplest workaround I’ve found to delete those entries was to set their EventType field value to 0, as they would be master entries, not recurring event exceptions. After this change, it was possible to delete the items. This kind of hack is also included in the DeleteIncludingRecurringEventExceptions extension method above.

Although the code I provided here seems to do the job and not pollute the Recycle Bin any more, if you have a really large number of items to delete, for performance reasons I still would prefer the bulk deletion of the events, or you should write your own solution to select all of the recurring event exceptions in the first step, and deleting them before you delete the main entries in the second step. I don’t think it would be a great idea to run a separate CAML query for each recurring events in your calendar.

Advertisements

July 27, 2018

A specific case of "The file name you specified is not valid or too long" error in SharePoint Explorer view

Filed under: SP 2013 — Tags: — Peter Holpar @ 21:07

Recently I found a document library in one of our SharePoint 2013 environments, that produced an odd behavior. I describe the symptoms and the solution below.

Symptoms

Problem with uploading file into a document library

I opened the Site Assets document library in a web site of a SharePoint 2013 installation (I don’t think the error is specific to this special kind of document libraries or SharePoint version, but I document it for the sake of completeness), and created a folder called ‘js’. The folder was created without any problem. I wanted to copy a .js file into the folder, but received the error message “The file name you specified is not valid or too long”. There was of course no problem with the length of the file nor with the name itself. I was simply not able to upload any file into that specific library, independent of the folder or the file name and extension. See below an error message for a text file. Clicking “Try Again” was unsuccessful.

image

I was able to upload files into other document libraries of the same site, as well as into libraries in other sites, so I assumed the problem is related to the library itself, but to be sure, I restarted first the WebClient service in Windows, then the whole Explorer process (including the desktop), finally the operating system itself, but all of them without any result (except losing lot of time).

I observed, however, that the file was uploaded despite of the error message. It appeared both in Windows Explorer as well as in the browser (I was able to open it!), at least till I clicked “Cancel” in the dialog box above. Then the file got deleted and disappeared, as it would have not been uploaded at all. Very strange.

I created a network trace using Fiddler to investigate the steps of the process. I include the HTTP response codes and optionally further information in brackets, unsuccessful steps are written in red:

PROPFIND /YourSite/SiteAssets (207)
PROPFIND / (207)
PROPFIND /YourSite/SiteAssets/YourFile.txt (404)
PROPFIND /YourSite/SiteAssets (207)
PUT /YourSite/SiteAssets/YourFile.txt (In the request we send the X-MSDAVEXTLockTimeout header with a value of Second-3600 and do not include the file in the request body, it indicates we acquire only a lock. The response includes these headers: X-MSDAVEXTLockTimeout: Second-3600; Lock-Token: opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z; X-MS-File-Checked-Out: 1. The HTTP response code is 201)
HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt We include the lock token we acquired earlier: Lock-Token: <opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>, another header X-MSDAVEXT: PROPPATCH and the file to be uploaded in the request body. The HTTP response code is 500, the response includes the header X-MSDAVEXT_Error: 589829; The%20URL%20%27SiteAssets%2fYourFile%2etxt%27%20is%20invalid%2e%20%20It%20may%20refer%20to%20a%20nonexistent%20file%20or%20folder%2c%20or%20refer%20to%20a%20valid%20file%20or%20folder%20that%20is%20not%20in%20the%20current%20Web%2e)
HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt (It’s not clear what happens in this step, we try to upload the file again, but with another headers this time, like If: (<opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>) and there is no X-MSDAVEXT. The file to be uploaded is included again in the request body. The HTTP response code is 500, the response includes the header X-MSDAVEXT_Error value as earlier.)
PROPFIND /YourSite/SiteAssets (207)
PROPFIND /YourSite/SiteAssets/desktop.ini HTTP/1.1 (404)

After clicking “Cancel”:

HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt HTTP/1.1 (This time we send only a DAV property update request, the file is not included in the request body. The HTTP response code is 500, the response includes the header X-MSDAVEXT_Error value as earlier.)
HEAD /YourSite/SiteAssets/YourFile.txt (200)
PUT /YourSite/SiteAssets/YourFile.txt HTTP/1.1 (Another put request with the header If: (<opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>) but without the file in the request body. The HTTP response code is again 500, the response includes the header X-MSDAVEXT_Error value as earlier.)
UNLOCK http://YourServer/YourSite/SiteAssets/MailAdresses.txt (we try to unlock the file, including the header Lock-Token: <opaquelocktoken:{D126A8C0-4DA2-4BD1-919F-566A6B68B507}20180725T072618Z>. The HTTP response code is 412)

PROPFIND /YourSite/SiteAssets/YourFile.txt (207)
DELETE http://YourServer/YourSite/SiteAssets/YourFile.txt (We delete the file. The HTTP response code is 204)

In the corresponding ULS log entries I found these lines (all entry with Area: SharePoint Foundation, Category: Database; Level: High):

System.Data.SqlClient.SqlException (0x80131904): Parameter ‘@tp_Version’ was supplied multiple times. at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows) at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more) at System.Data.SqlClient.SqlDataRea… …der.TryNextResult(Boolean& more) at System.Data.SqlClient.SqlDataReader.NextResult() at Microsoft.SharePoint.SPSqlClient.ExecuteQueryInternal(Boolean retryfordeadlock) at Microsoft.SharePoint.SPSqlClient.ExecuteQuery(Boolean retryfordeadlock) ClientConnectionId:0ed39f0d-b333-43fd-90e3-421a19b57441 Error Number:8143,State:1,Class:16 ExecuteQuery failed with original error 0x80131904 SQL error code from last error 8143 – Parameter ‘@tp_Version’ was supplied multiple times.

Creating new files into a document library

It is the same library as earlier, but instead of copying a file into it, I have tried to create a new file this time. I’ve selected New / Text Document in Windows Explorer, have renamed it to test.txt, opened it in Notepad, added some content and saved it. The file was saved, and was available in Windows Explorer as well in the browser, but a warning appeared in the low right corner of the desktop, saying:

Delayed Write Failed
Windows was unable to save all the data for the file \YourServer\YourSite\SiteAssets\test.txt. The data has been lost. This error may be caused by a failure of your computer hardware or network connection. Please try to save this file elsewhere.

image

I found a page about that issue but it said really nothing, and was not right either, as the file was saved in my case, despite of the warning.

I’ve found an entry in Event Viewer (Windows Logs / Application having Source: WebClient; Level: Warning) that probably belongs to the error above:

The description for Event ID 14903 from source WebClient cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
\YourServer\YourSite\SiteAssets\New Text Document.txt
123

It wasn’t really helpful too.

The solution

What really helped was the ULS log I’ve included above, especially the part:

(0x80131904): Parameter ‘@tp_Version’ was supplied multiple times.

I was sure I hade already the same or a very similar issue (eventually with another parameter name), yet with SharePoint 2010, and at that time I verified the statement with SQL Profiler and found, that the parameter is sent really twice in the query. Unfortunately (or fortunately?), it was a long time ago, and I have not remembered the solution. But a quick search on the web helped, and I found this forum entry.

Although related to an other field (Author), the answer states, that a similar problem was caused by a field schema corruption. So I decided to check the schema of the field that belongs to the tp_Version parameter (that is the owshiddenversion field):

$web = Get-SPWeb http://YourServer/YourSite
$list = $web.Lists[‘Site Assets’]
$list.Fields.InternalName
$list.Fields[‘owshiddenversion’]
$versionField = $list.Fields[‘owshiddenversion’]
$versionField.SchemaXml

The result was:

<Field ID="{d4e44a66-ee3a-4d02-88c9-4ec5ff3f4cd5}" Name="owshiddenversion" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="owshiddenversion" Group="_Hidden" ColName="tp_Version" RowOrdinal="0" Hidden="TRUE" ReadOnly="TRUE" Type="Integer" SetAs="owshiddenversion" DisplayName="owshiddenversion" Sealed="FALSE" Version="1" />

I then compared the results with the field schema of another document library, where the upload works as it should:

$list = $web.Lists[‘Documents’]
$versionField = $list.Fields[‘owshiddenversion’]
$versionField.SchemaXml

The result was this time:

<Field ID="{d4e44a66-ee3a-4d02-88c9-4ec5ff3f4cd5}" ColName="tp_Version" RowOrdinal="0" Hidden="TRUE" ReadOnly="TRUE" Type="Integer" SetAs="owshiddenversion" Name="owshiddenversion" DisplayName="owshiddenversion" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="owshiddenversion" FromBaseType="TRUE"/>

You can see, that – exactly as the forum answer suggests – the attribute FromBaseType="TRUE" is missing from the schema XML. To fix the difference, I run this script on the corrupted library:

$xml = [xml]$versionField.SchemaXmlWithResourceTokens
$xml.Field.SetAttribute(‘FromBaseType’,’TRUE’)
$versionField.SchemaXml = $xml.OuterXml
$versionField.Update()

By running the script the difference in the field schema  XML was fixed, and there was no more problem with copying to and creating files in the library.

July 1, 2018

HttpRequest.Url contains always the full URL of the current request, doesn’t it?

Filed under: Bugs, SP 2013 — Tags: , — Peter Holpar @ 20:53

Recently we had to create a Wellcome menu extension in SharePoint to make a custom application page available from all of the web site context, like standard lists and pages as well as standard and custom application pages. This custom application page provides its own functionality (irrelevant to the problem described in the post), and after the user performed the task on the page, and submitted it via a button click, he should be returned to the original page, where he invoked the custom page from the menu. As the condition, if the menu item should or should not be displayed for a specific user, depends on some complex criteria (omitted from the code snippets in the post), we decided to implement the menu item on the server side by our custom MenuItemTemplate.

The CustomAction definition sets GroupId as PersonalActions and Location as Microsoft.SharePoint.StandardMenu to have the menu item in the Wellcome menu. The ControlAssembly and ControlClass attributes identify the class, in that we implemented the solution.

  1. <?xml version="1.0" encoding="utf-8"?>
  2. <Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  3.   <CustomAction
  4.       Id="YourCustomIdAction"
  5.       GroupId="PersonalActions"
  6.       Location="Microsoft.SharePoint.StandardMenu"
  7.       ControlAssembly="$SharePoint.Project.AssemblyFullName$"
  8.       ControlClass="YourNamespace.DisplayCustomMenuItem"
  9.     >
  10.   </CustomAction>
  11. </Elements>

In the CreateChildControls method of our class we create a new MenuItemTemplate instance, set its properties and finally add it to the Controls collection. See the ClientOnClickNavigateUrl property, that we set by invoking the static GetRedirectionUrl method of our helper class.

  1. public class DisplayCustomMenuItem : WebControl
  2. {
  3.     protected override void CreateChildControls()
  4.     {
  5.         MenuItemTemplate item = new MenuItemTemplate();
  6.         item.Description = "Description of your menu extension";
  7.         item.Sequence = 1000;
  8.         item.Text = "Do something";
  9.         item.ClientOnClickNavigateUrl = Helper.GetRedirectionUrl();
  10.  
  11.         Controls.Add(item);
  12.     }
  13. }

The original version of the static GetRedirectionUrl method was this one:

  1. public static string GetRedirectionUrl()
  2. {
  3.     SPContext ctx = SPContext.Current;
  4.     HttpContext context = HttpContext.Current;
  5.     string absUrl = context.Request.Url.AbsoluteUri;
  6.     string result = string.Format("{0}{1}?{2}={3}", ctx.Web.Url, Constants.RedirectionPagePath, QueryString.Source, HttpUtility.UrlEncode(absUrl));
  7.  
  8.     return result;
  9. }

Constants and QueryString are static classes. Constants.RedirectionPagePath stores the site relative path of our custom application page, like "/_layouts/15/OurCustomFolder/CustomApplicationPage.aspx", QueryString.Source is the name of the query string parameter (like ‘CustomSource’) variable we used to specify the URL of the page, where the user selected the menu item, and to which the application page should redirect after the user finished his task.

Our application page is inherited from the Microsoft.SharePoint.WebControls.LayoutsPageBase base class. The relevant part is the Click event handler of our button. We read the value of original URL from the query string parameter QueryString.Source, and redirect the response accordingly:

  1. protected void DoSomething_Click(object sender, EventArgs e)
  2. {
  3.     // do something here
  4.  
  5.     // redirect to the original page
  6.     var sourceUrl = this.Request.QueryString[QueryString.Source];
  7.     if (!string.IsNullOrEmpty(sourceUrl))
  8.     {
  9.         LoggingService.LogMessage("Redirecting request to page: '{0}'", sourceUrl);
  10.         Response.Redirect(sourceUrl);
  11.     }
  12. }

We activated the feature in a site collection like http://YourSharePoint/SomeSiteCollection. While testing the application, we found that the redirection worked as expected for all of the list views and standard (web part or wiki) pages, but malfunctioned for any application pages, either for standard or custom ones. For example, if the start page, where the user selected the custom menu item from the Welcome menu, was the Site Settings page of the root web site or any other sub web site of our site collection, having a URL like http://YourSharePoint/SomeSiteCollection/_layouts/15/settings.aspx, our custom application was displayed, but after clicking the button, the request was redirected instead of the original Site Setting page to the Site Setting page of the root site collection of the web application (that means, to URL http://YourSharePoint/_layouts/15/settings.aspx).

After debugging the solution in the context of various site collections, it turned out, that the value of the HttpContext.Current.Request.Url for application pages refers always to the page in the context of the root site collection of the web application (that means a URL beginning with http://YourSharePoint/_layouts/15). Of course, this result is not what you expect if your site is not the root one.

After even more debugging and browsing several properties of the current SPContext and HttpContext objects in run-time, I have finally found a value in the HttpContext.Items collection (the one with the key Microsoft.SharePoint.SPGlobal.GetVTIRequestUrl), that contained exactly the same URL value that corresponded the site specific URL of the application pages, the one we needed to achieve our original goal.

So I rewrote the GetRedirectionUrl method to check if the URL returned by the HttpContext.Current.Request.Url property matches to the URL of the current web site (as it is returned by SPContext.Current.Web.Url) and if the values differ, use the value stored in the entry in the HttpContext.Items collection having the Microsoft.SharePoint.SPGlobal.GetVTIRequestUrl key (as long as it is available). The code snippet below displays the updated version of the method, that finally performs as we expected:

  1. public static string GetRedirectionUrl()
  2. {
  3.     SPContext ctx = SPContext.Current;
  4.     HttpContext context = HttpContext.Current;
  5.     string absUrl = context.Request.Url.AbsoluteUri;
  6.     string webUrl = ctx.Web.Url;
  7.     string vtiRequestUrlKey = "Microsoft.SharePoint.SPGlobal.GetVTIRequestUrl";
  8.     Uri vtiRequestUrl = context.Items.Contains(vtiRequestUrlKey) ? context.Items[vtiRequestUrlKey] as Uri : null;
  9.     string requestUrl = ((absUrl.IndexOf(webUrl) == 0) || (vtiRequestUrl == null)) ? absUrl : vtiRequestUrl.AbsoluteUri;
  10.     string result = string.Format("{0}{1}?{2}={3}", webUrl, Constants.RedirectionPagePath, QueryString.Source, HttpUtility.UrlEncode(requestUrl));
  11.  
  12.     return result;
  13. }

Lesson learned: be aware that the HttpContext.Current.Request.Url may return the wrong value for application pages, and use the value of the entry with the Microsoft.SharePoint.SPGlobal.GetVTIRequestUrl key in the HttpContext.Items collection instead.

The sub-webs delivered to you are dirty

Filed under: PowerShell, SP 2013 — Tags: , — Peter Holpar @ 14:39

Recently I had to perform a simple administrative task: a SharePoint website had several sub-webs, each of them having unique permissions. Our goal was to reset the permissions to be inherited from the parent site. So I created a simple PowerShell script to achieve the goal:

$rootWeb = Get-SPWeb ‘http://YourSharePointSite/SomeSite&#8217;
$rootWeb.Webs | % { $_.ResetRoleInheritance() }

Note: Although I use the Webs property of the SPWeb object overall in this post to illustrate the problem and later the solution, the very same applies to the GetSubwebsForCurrentUser method of the SPWeb object as well.

To my greatest surprise, I received this error message for each of the sub-web sites:

Exception calling "ResetRoleInheritance" with "0" argument(s): "There are uncommitted changes on the SPWeb object, call SPWeb.Update() to commit the changes before calling this method."

To understand the source of the exception, I followed the call-chain of the SPWeb.ResetRoleInheritance method using Reflector. All of the methods and classes mentioned in the post are declared in the Microsoft.SharePoint assembly in the Microsoft.SharePoint namespace.

First, the SPWeb.ResetRoleInheritance() method invokes the virtual SPSecurableObject.ResetRoleInheritance(), that invokes the internal SPSecurableObjectImpl.ResetRoleInheritance() method, that finally calls the private SPSecurableObjectImpl.RevertRoleInheritance(bool copyRoleAssignments, bool clearSubScopes) method, where the exception get thrown.

The value of the $StackTrace variable in PowerShell confirmed the result of my research:

   at Microsoft.SharePoint.SPSecurableObjectImpl.RevertRoleInheritance(Boolean copyRoleAssignments, Boolean clearSubScopes)
   at Microsoft.SharePoint.SPWeb.ResetRoleInheritance()
   at CallSite.Target(Closure , CallSite , Object )

The SPSecurableObjectImpl.RevertRoleInheritance method contains this condition:

if ((this.m_objectType == SPObjectType.Web) && this.m_web.IsDirty)
{
    throw new InvalidOperationException(SPResource.GetString("SPWebHasUnCommittedChange", new object[0]));
}

It means, the internal IsDirty property of the SPWeb class is checked to see, if there is any uncommitted change in the SPWeb instance. It is common, that your SPWeb instance gets dirty after you change some of its properties, but in our case, we apparently have not change anything, we still get the complain about “dirtiness” our web.

Let’s see, how to check if our SPWeb instance is dirty or not, and find some kind of workaround.

Forget the PowerShell example above for a while, and switch to C#. In this case, the base version of the code looks like this:

  1. using (SPSite site = new SPSite("http://YourSharePointSite/SomeSite&quot;))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         foreach (SPWeb subWeb in web.Webs) // or the same with webs.GetSubwebsForCurrentUser()
  6.         {
  7.             try
  8.             {
  9.                 subWeb.ResetRoleInheritance();
  10.             }
  11.             finally
  12.             {
  13.                 subWeb.Dispose();
  14.             }
  15.         }
  16.     }
  17. }

Although the IsDirty method of the SPWeb is declared as private, we can access it via Reflection, as we did it in the extension method below:

  1. static class Extensions
  2. {
  3.     public static bool IsDirty(this SPWeb web)
  4.     {
  5.         var result = false;
  6.  
  7.         var pi_isDirty = typeof(SPWeb).GetProperty("IsDirty", BindingFlags.NonPublic | BindingFlags.Instance);
  8.         result = (bool)pi_isDirty.GetValue(web);
  9.  
  10.         return result;
  11.     }
  12. }

Having our extension method, we can dump out easily, if  the root web site and its sub-webs are dirty or not.

  1. using (SPSite site = new SPSite("http://YourSharePointSite/SomeSite&quot;))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         Console.WriteLine("Web '{0}' is dirty: '{1}'", web.Url, web.IsDirty());
  6.         foreach (SPWeb subWeb in web.Webs) // or the same with webs.GetSubwebsForCurrentUser()
  7.         {
  8.             try
  9.             {
  10.                 Console.WriteLine("Web '{0}' is dirty: '{1}'", subWeb.Url, subWeb.IsDirty());
  11.                 subWeb.ResetRoleInheritance();
  12.             }
  13.             finally
  14.             {
  15.                 subWeb.Dispose();
  16.             }
  17.         }
  18.     }
  19. }

The result shows, that the parent site is not dirty, but all of it sub-webs (returned either by the Webs property or the GetSubwebsForCurrentUser method) are all dirty.

There are two possible workarounds for the issue. We should either call the Update method of the SPWeb instance before invoking the ResetRoleInheritance method, thus clearing the IsDirty flag, or if we don’t want to commit any possible changes, we can create another, clear SPWeb instance from scratch based on the ID or the Url of the original SPWeb object, and invoke the ResetRoleInheritance method on the new instance. The code sample below illustrates both of these options:

  1. using (SPSite site = new SPSite("http://YourSharePointSite/SomeSite&quot;))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         Console.WriteLine("Web '{0}' is dirty: '{1}'", web.Url, web.IsDirty());
  6.  
  7.         foreach (SPWeb subWeb in web.Webs) // or the same with webs.GetSubwebsForCurrentUser()
  8.         {
  9.             try
  10.             {
  11.                 Console.WriteLine("Web '{0}' is dirty: '{1}'", subWeb.Url, subWeb.IsDirty());
  12.                 // option 1
  13.                 subWeb.Update();
  14.                 Console.WriteLine("Web '{0}' is dirty: '{1}'", subWeb.Url, subWeb.IsDirty());
  15.                 subWeb.ResetRoleInheritance();
  16.                 //// option 2
  17.                 //using (SPWeb subWebNew = site.OpenWeb(subWeb.ID)) // or site.OpenWeb(subWeb.Url)
  18.                 //{
  19.                 //    Console.WriteLine("Web '{0}' is dirty: '{1}'", subWebNew.Url, subWebNew.IsDirty());
  20.                 //    subWebNew.ResetRoleInheritance();
  21.                 //}
  22.             }
  23.             finally
  24.             {
  25.                 subWeb.Dispose();
  26.             }
  27.         }
  28.     }
  29. }

After this detour into C#, let’s go back to our original PowerShell sample. Although there are no extension methods in PowerShell, we can define a helper function to query and display the value of the IsDirty property, and we can apply both of the above workarounds to “clear” or web instance as well:

  1. function IsDirty($web) {
  2.     $pi = [Microsoft.SharePoint.SPWeb].GetProperty("IsDirty", [Reflection.BindingFlags] "NonPublic,Instance")
  3.     $isDirty = $pi.GetValue($web)
  4.     return $isDirty
  5.     Write-Host Web $($web.Url) is dirty $isDirty
  6. }
  7.  
  8. $rootWeb = Get-SPWeb 'http://YourSharePointSite/SomeSite&#039;
  9. Write-Host Web $($rootWeb.Url) is dirty $($rootWeb.IsDirty)
  10. $rootWeb.Webs | % {
  11.     $web = $_
  12.     IsDirty $web
  13.     # option 1
  14.     $web.Update()
  15.     IsDirty $web
  16.     $web.ResetRoleInheritance()
  17.     ## option 2
  18.     #$webClear = Get-SPWeb $web.Url
  19.     #IsDirty $webClear
  20.     #$webClear.ResetRoleInheritance()
  21. }

But wait! Is there really nothing in PowerShell like extension methods in C#? Couldn’t we extend or object somehow to be able to write nicer code? About this theme and much more plan I write in a later post.

 

April 30, 2018

Faking feature activation properties

Filed under: Features, Reflection, SP 2013, Tips & Tricks — Tags: , , , — Peter Holpar @ 21:19

Assume you have a feature receiver in your SharePoint project. You would like to perform multiple actions if the feature is activated, so you organize your code according to the actions into static methods of a helpers class, like shown in the code snippet below:

  1. [Guid("8cb098ae-2017-4fff-8a53-b315abb85d79")]
  2. public class YourFeatureReceiver : SPFeatureReceiver
  3. {
  4.     public override void FeatureActivated(SPFeatureReceiverProperties properties)
  5.     {
  6.         SomeHelperClass.DoSomethingOnTheSite(properties);
  7.         SomeHelperClass.DoSomethingOtherOnTheSite(properties);
  8.         SomeHelperClass.AndFinallyDoThat(properties);
  9.     }
  10. }

Note: In this case, it is a site level feature, and we handle the FeatureActivated event, but you can apply the same technique for other kinds of feature receivers and events as well.

A helper method has the following signature, and contains code like this one:

  1. public static void DoSomethingOnTheSite(SPFeatureReceiverProperties properties)
  2. {
  3.     SPSite site = properties.Feature.Parent as SPSite;
  4.     if (site != null)
  5.     {
  6.         // do something here
  7.     }
  8. }

You would like to test the functionality of your method one-by-one from a console application, without actually having to deploy your SharePoint solution an activate your feature each time again (of course, you need at least an initial deployment of the whole solution). In my case, the helper class is included in a separate assembly, but even if it is the assembly of your SharePoint project, you should only deploy the new version of the assembly into the Global Assembly Cache (GAC).

As you can see in the method above, we use the Feature property of the SPFeatureReceiverProperties class we received in the properties parameter. Of course, the Feature property is read-only, so we need some tricks, to be able to pass the SPFeatureReceiverProperties parameter populated with the correct SPFeature in its Feature property to the method. We create first a new instance of the SPFeatureReceiverProperties class, than query the site for the feature based on its ID (se my important notice about this ID after the code block!). We can inject this SPFeature instance into our formerly crated SPFeatureReceiverProperties instance using Reflection, via the internal SetFeature method of the SPFeatureReceiverProperties class. Finally, we can invoke our helper method with the already-populated properties variable.

  1. using (SPSite site = new SPSite(yourSiteUrl))
  2. {
  3.     var properties = new SPFeatureReceiverProperties();
  4.  
  5.     var yourFeatureId = new Guid("51bf2a39-b527-46cf-abd6-39aaf1dcd19b");
  6.  
  7.     var feature = site.Features.FirstOrDefault(f => f.DefinitionId == yourFeatureId);
  8.     if (feature != null)
  9.     {
  10.         MethodInfo mi_setFeature = typeof(SPFeatureReceiverProperties).GetMethod("SetFeature",
  11.             BindingFlags.NonPublic | BindingFlags.Instance);
  12.  
  13.         if (mi_setFeature != null)
  14.         {
  15.             mi_setFeature.Invoke(properties, new object[] { feature });
  16.             SomeHelperClass.DoSomethingOnTheSite(properties);
  17.         }
  18.     }
  19. }

Note: It is important to notice, that the Guid we used in the feature-lookup above, comparing it to the DefinitionId property, is not the same, as the Guid we saw earlier in the feature receiver code snippet. The latter one is used only by Visual Studio during the deployment process to find up the correct ReceiverClass for the event receiver. The Guid we need, the ID of the feature is available on the Manifest tab of the feature as illustrated below:

image

Applying this technique made our development and testing process a lot faster.

Creating statistics about web part usage from the SharePoint content database

Filed under: PowerShell, Reflection, SP 2013, SQL, Web part — Tags: , , , — Peter Holpar @ 21:08

Recently I had to create some statistics about SharePoint web site customizations, like on which pages are there Script Editor Web Parts, or Content Editor Web Parts, etc. I knew I could and probably should have done it by iterating through all web sites, all pages and then looking up the web parts on each page using SPLimitedWebPartManager class, but I was aware, the same information should be available via the content database as well, making it possible to query the info much easier and faster, although unsupported. In this post I describe, how you can do it, but use the solution at your own risk.

The web part information is stored in the AllWebParts table, the information about the pages in the AllDocs table. I joined these tables together for the first report about the Script Editor Web Parts.

SELECT AD.DirName + ‘/’ + AD.LeafName as PageUrl, AWP.tp_ZoneID as ZoneId, AWP.tp_PartOrder as WebPartOrder, AWP.tp_Class AS WebPartClass
FROM
AllWebParts AWP (nolock)
INNER JOIN AllDocs (nolock) AD ON AWP.tp_SiteId = AD.SiteId AND AWP.tp_PageUrlID = AD.Id
WHERE tp_Class LIKE ‘%ScriptEditorWebPart’

Next, I was to create a report about the Content Editor Web Parts, using a filter like:

WHERE tp_Class = ‘%ContentEditorWebPart’

However, no result found, although I was pretty sure, there are a lot of them in our web site. How is it possible?

To test it further, I’ve included a Script Editor Web Part and a Content Editor Web Part on the AllItems.aspx page of the Tasks list in one of our sub-site, and created a new query with the filter below:

WHERE DirName LIKE ‘%site/subsite/Lists/Tasks%’
AND LeafName LIKE ‘%AllItem%’

This was the result:

image

As you see, the Script Editor Web Part is there, and you see two further web parts (they should be the Content Editor Web Part and the XsltListViewWeb part, that was originally on the page and is responsible to display the task items in the list), however both of them with a NULL value in the WebPartClass column. What should it mean?

I have studied the structure of the AllWebParts table and the relations of its fields further, and found that there are two fields (tp_Class and tp_Assembly) that are always populated for the records, where the WebPartClass is not NULL, and there is a field called tp_WebPartTypeId – populated for each entries, even for those, where the WebPartClass , tp_Class and tp_Assembly fields are empty – that we could eventually use to find the matching web parts. But how? I made a search for ‘WebPartTypeId’ using .NET Reflector, and found the internal class Microsoft.SharePoint.WebPartPages.WebPartTypeInfo, having a private static method called GetWellKnownTypeIdDictionary that returns a Dictionary<Guid, Type> mapping Guids (WebPartTypeIds) to the actual web part type. Remark: The Guids in the WebPartTypeId are actually created from the MD5 hash of the bytes of the joined full assembly name and web part class name, see the internal static  GetTypeIdUnsafe(MD5HashProvider md5Provider, string typeFullName, string assemblyName) method of the internal sealed class Microsoft.SharePoint.ApplicationRuntime.SafeControls.

image

To support those so called well-known types in my former SQL-query, I wrote a short PowerShell script that invokes the private static GetWellKnownTypeIdDictionary method of the internal WebPartTypeInfo class, and emits the resulting Dictionary to a text file I can use to extend my query:

  1. $webPartTypeInfoType = [System.Type]::GetType('Microsoft.SharePoint.WebPartPages.WebPartTypeInfo, Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c')
  2. $mi_GetWellKnownTypeIdDictionary = $webPartTypeInfoType.GetMethod('GetWellKnownTypeIdDictionary', [Reflection.BindingFlags]'NonPublic, Static')
  3. $wellKnownTypeIdDictionary = $mi_GetWellKnownTypeIdDictionary.Invoke($null, $null)
  4.  
  5. $wpTypes = $wellKnownTypeIdDictionary.Keys | % { "INSERT INTO @WPTypes VALUES ('$_', '$($wellKnownTypeIdDictionary[$_].Assembly.FullName)', '$($wellKnownTypeIdDictionary[$_].FullName)')" }
  6. Set-Content -Path 'C:\Data\WPTypes.txt' -Value $wpTypes

And that is already the extended version of the SQL query:

  1. DECLARE @WPTypes TABLE
  2.    (
  3.      Id uniqueidentifier NOT NULL,
  4.      AssemblyName varchar(500),
  5.      ClassName varchar(100)
  6.    )
  7.  
  8. INSERT INTO @WPTypes VALUES ('8e20cf70-0fd5-1e08-9972-38f63a6bd59a', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ImageWebPart')
  9. INSERT INTO @WPTypes VALUES ('ba009853-eac3-16c8-9094-a8834485ad33', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.DataFormWebPart')
  10. INSERT INTO @WPTypes VALUES ('83216ab2-cd0e-e9fc-fc5e-6a8f3b21c37b', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.DataViewWebPart')
  11. INSERT INTO @WPTypes VALUES ('42fddde2-e0cf-c8ab-48b7-db1fcac0a917', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ListFormWebPart')
  12. INSERT INTO @WPTypes VALUES ('05d0fd94-372a-5ee7-b480-ccb8f9cd2c23', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ListViewWebPart')
  13. INSERT INTO @WPTypes VALUES ('aef28218-44f8-0538-9805-4842c0e62811', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.XsltListFormWebPart')
  14. INSERT INTO @WPTypes VALUES ('a6524906-3fd2-ee4e-23ee-252d3c6e0dc9', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.XsltListViewWebPart')
  15. INSERT INTO @WPTypes VALUES ('0c6143a7-d68b-bade-e0ef-2c4d01182b0c', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.BlogAdminWebPart')
  16. INSERT INTO @WPTypes VALUES ('afef48e1-8f94-eb71-03a6-ffceb685306a', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.BlogMonthQuickLaunch')
  17. INSERT INTO @WPTypes VALUES ('4c06cea2-364f-47e3-e1d7-08d53f441157', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ContentEditorWebPart')
  18. INSERT INTO @WPTypes VALUES ('e6047383-438e-ed87-1a93-f1ff71729044', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.TitleBarWebPart')
  19. INSERT INTO @WPTypes VALUES ('707c1e73-0b3d-898b-c755-01621802ab8c', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.SilverlightWebPart')
  20. INSERT INTO @WPTypes VALUES ('28c23aec-2537-68b3-43b6-845b13cea19f', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ErrorWebPart')
  21. INSERT INTO @WPTypes VALUES ('8d6034c4-a416-e535-281a-6b714894e1aa', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ErrorWebPart')
  22. INSERT INTO @WPTypes VALUES ('8e814083-396a-e7d1-148b-316e3a7283f7', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ErrorWebPart')
  23. INSERT INTO @WPTypes VALUES ('e6377261-6920-bbfe-501f-fda7a61db10f', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ErrorWebPart')
  24. INSERT INTO @WPTypes VALUES ('8efd140d-eae9-5feb-06e3-f771842d2e43', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.ErrorWebPart')
  25. INSERT INTO @WPTypes VALUES ('b3294a07-46bf-e661-d036-10670590bbd3', 'Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', 'Microsoft.SharePoint.WebPartPages.SPUserCodeWebPart')
  26.  
  27. SELECT AD.DirName + '/' + AD.LeafName as PageUrl, AWP.tp_ZoneID as ZoneId, AWP.tp_PartOrder as WebPartOrder, ISNULL(AWP.tp_Class, WPT.ClassName) AS WebPartClass
  28. FROM AllWebParts AWP (nolock)
  29. INNER JOIN AllDocs AD (nolock) ON AWP.tp_SiteId = AD.SiteId AND AWP.tp_PageUrlID = AD.Id
  30. LEFT JOIN @WPTypes WPT ON AWP.tp_WebPartTypeId = WPT.Id
  31. WHERE ISNULL(AWP.tp_Class, WPT.ClassName) LIKE '%ContentEditorWebPart'

Of course, you can change the conditions of the query as you like, for example, you can restrict it to two web part type, like:

WHERE ISNULL(AWP.tp_Class, WPT.ClassName) IN (‘Microsoft.SharePoint.WebPartPages.ScriptEditorWebPart’, ‘Microsoft.SharePoint.WebPartPages.ContentEditorWebPart’)

There are a few more columns in the AllWebParts table, that you eventually would include either in the SELECT statement or in its WHERE clause, these are:

  • tp_IsIncluded: The web part is displayed on the page, if the value is 1 (default). If you close (not delete!) a web part, the value is 0. Deleted web parts are removed from the table.
  • tp_Deleted: Assume you have a list with some pages that includes web parts, like view pages including XsltListViewWebPart instances. The web part entries in the AllWebParts table have a value of 0 at this stage. If you delete the list, these values change to 1. The web part entries will be kept in the table even after deleting the list from the first (user) level Recycle Bin, and removed only after the list is deleted from the second (site collection) level Recycle Bin.
  • tp_ListId: This is a field that is populated for list-related built-in web parts, like XsltListViewWebPart. You can look up the related list and web instances by joining the Lists and Webs views in your query respectively, as shown below (this time I omit the declaration of the @WPTypes variable and its population with value for the sake of brevity, but of course, you need it this time either):

SELECT L.tp_Title as ListTitle, W.FullUrl AS WebUrl, AD.DirName + ‘/’ + AD.LeafName as PageUrl, AWP.tp_ZoneID as ZoneId, AWP.tp_PartOrder as WebPartOrder, ISNULL(AWP.tp_Class, WPT.ClassName) AS WebPartClass, tp_ListId
FROM AllWebParts AWP (nolock)
INNER JOIN AllDocs AD (nolock) ON AWP.tp_SiteId = AD.SiteId AND AWP.tp_PageUrlID = AD.Id
LEFT JOIN @WPTypes WPT ON AWP.tp_WebPartTypeId = WPT.Id
LEFT JOIN Lists L (nolock) ON AWP.tp_SiteId = L.tp_SiteId AND AWP.tp_ListId = L.tp_ID
LEFT JOIN Webs W (nolock) ON AWP.tp_SiteId = W.SiteId AND L.tp_WebId = W.Id
WHERE ISNULL(AWP.tp_Class, WPT.ClassName) LIKE ‘%ListViewWebPart’

By including the list title or the web URL in the WHERE clause (or the ID of the list or the web if you wish) you can further limit the items returned by the query.

If there are records returned with NULL in the ListTitle and WebUrl columns it means typically that the list was deleted, but yet available in the Recycle Bin. See my comments regarding the tp_Deleted field above. Note, that despite the name of the FullUrl column in the Web view, it is actually a server relative URL.

I hope this overview has helped you to better understand what and how is stored in these tables of the SharePoint content database, as well, how the “magical” IDs of the well-known web part types do fit into the whole picture.

March 26, 2018

Querying SharePoint list items including their attachment in a single request from the JavaScript Client Object Model

Filed under: CAML, ECMAScript Client Object Model, JSCOM, SP 2013 — Tags: , , , — Peter Holpar @ 23:31

Assume there is a SharePoint list including a few hundreds of items, each item might have a few attachments, but there might be items without attachments as well. We need to display only some of the items (filtered dynamically based on specific conditions) for the users on a web page. We use already the JavaScript client object model on that page, so it would be ideal to keep this technology, instead of involving another one (like REST or SharePoint web services) just because this task.

I found several useful pages while looking for a solution for my problem, like this one about using CAML from the client object model, although some of the statements in the sections “Include attachment URLs” and “Limitations” turned to be a bit misleading later.

Few posts and forum answers suggested to query the items first using a CAML query, then read the attachments for each matching item in subsequent requests individually. That would not perform very well if there are a lot of matching item and is generally not a nice solution. Another suggestion I often found is to query the items with a CAML query (just like in the previous case), then query all of the attachments (they are stored in a folder called Attachments n the list with a separate subfolder, like 1, 2, etc. for each list items) in a second query, then match the attachments files to items based on the relative path of the attachments (attachment for the list item with ID 1 are in the folder /Attachments/1, etc.). If you have a lot of items having a few attachments for each item, that might have performance problem as well, and the implementation itself is challenging.

Vadim Gremyachev posted a solution using the managed client object model with a single request only on SharePoint Stack Exchange. In his solution, he uses the following line to create an execute the CAML query:

var items = list.GetItems(CamlQuery.CreateAllItemsQuery());

The static CreateAllItemsQuery method of the CamlQuery object is only an easy way to create a CAML query having the RecursiveAll view scope, it means, content of the subfolders (if there is any) would be searched through either.

public static CamlQuery CreateAllItemsQuery()
{
    return new CamlQuery { ViewXml = "<View Scope=\"RecursiveAll\">\r\n    <Query>\r\n    </Query>\r\n</View>" };
}

As we have a flat list, without any folders, I’ve slightly modified Vadim’s code first:

  1. var siteUrl = "http://YourSharePoint/site/subsite&quot;;
  2. using (var ctx = new ClientContext(siteUrl))
  3. {
  4.     var web = ctx.Web;
  5.     var list = web.Lists.GetByTitle("TitleOfYourList");
  6.     // Vadim's version fo a list with a folder struture
  7.     //var items = list.GetItems(CamlQuery.CreateAllItemsQuery());
  8.     // my version for a flat list without folders
  9.     var items = list.GetItems(new CamlQuery());
  10.     ctx.Load(items, icol => icol.Include(i => i["Title"], i => i.AttachmentFiles));
  11.     ctx.ExecuteQuery();
  12.  
  13.     //Print attachment Urls for list items
  14.     foreach (var item in items)
  15.     {
  16.         if (item.AttachmentFiles.Count > 0)
  17.         {
  18.             Console.WriteLine("{0} attachments:", item["Title"]);
  19.             foreach (var attachment in item.AttachmentFiles)
  20.             {
  21.                 Console.WriteLine(attachment.ServerRelativeUrl);
  22.             }
  23.         }
  24.         else
  25.         {
  26.             Console.WriteLine("No attachments were found for list item '{0}' ", item["Title"]);
  27.         }
  28.     }
  29. }

It resulted in the following request sent to the server, captured by Fiddler:

image

Note, that only the properties Title and AttachmentFiles were requested in the Include statement, and the ViewXml is empty.

We wanted to reproduce the same request using the JavaScript object model, but haven’t found any solution immediately ready to use, so first made a quick check, which alternative solutions are available in JavaScript, if we were not able to transform the managed code sample to JavaScript. Let’s see what we found.

I found a jQuery-based approach here, that utilizes the GetListItems method of the Lists SharePoint web service:

  1. $("table.ms-listviewtable tr:has(td.ms-vb2)").find("td:first").filter(function() {
  2.     return $(this).text().toLowerCase() == ID;
  3. }).html("<img src='" + url[1] + "' width=150 height=100 />");
  4.  
  5. $(document).ready(function() {
  6.     querySPWebServices();
  7. });
  8. function querySPWebServices() {
  9.     var soapEnv = "<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/'&gt; \\par         <soapenv:Body> \\par         <GetListItems xmlns='http://schemas.microsoft.com/sharepoint/soap/'&gt; \\par         <listName>Projects</listName> \\par         <viewFields> \\par         <ViewFields> \\par         <FieldRef Name='Title' /> \\par         <FieldRef Name='Body' /> \\par         <FieldRef Name='ID' /> \\par         <FieldRef Name='Attachments' /> \\par         </ViewFields> \\par         </viewFields> \\par         <query> \\par         <Query /> \\par         </query> \\par         <queryOptions> \\par         <QueryOptions> \\par         <IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls> \\par         </QueryOptions> \\par         </queryOptions> \\par         </GetListItems> \\par         </soapenv:Body> \\par         </soapenv:Envelope>";
  10.     $.ajax({
  11.         async: false,
  12.         url: "http://server.name.com/site/_vti_bin/lists.asmx&quot;,
  13.         type: "POST",
  14.         dataType: "xml",
  15.         data: soapEnv,
  16.         complete: processResult,
  17.         contentType: "text/xml; charset=\"utf-8\""
  18.     });
  19. };
  20.  
  21. function processResult(xData, status) {
  22.     $(xData.responseXML).find("z\\:row").each(function() {
  23.         if ($(this).attr("ows_Attachments") != 0) {
  24.             var url = $(this).attr("ows_Attachments").replace(/#/g, "").split(';');
  25.             var ID = $(this).attr("ows_ID");
  26.             $("table.ms-listviewtable tr:has(td.ms-vb2)").find("td:first").filter(function() {
  27.                 return $(this).text().toLowerCase() == ID;
  28.             }).html("<img src='" + url[1] + "' width=150 height=100 />");
  29.         };
  30.     });
  31. };

Another solution invoking the GetListItems method as well, but this time using SPServices can be found here:

  1. var p = $().SPServices({
  2.     operation: "GetListItems",
  3.     listName: "listname",
  4.     CAMLQueryOptions: "<QueryOptions>" +
  5.             "<IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls>" +
  6.         "</QueryOptions>"
  7. });
  8.   
  9. p.done(function() {
  10.   
  11.     $(p.responseXML).).SPFilterNode("z:row").each(function() {
  12.       
  13.         var attachments = [];
  14.   
  15.         var att = $(this).attr("ows_Attachments");
  16.         if(att !== "0") {
  17.             attachments = att.split(";#"); // Now you'll have an array of attachment URLs
  18.         } else {
  19.             // att will be "0", indicating that there are no attachments
  20.         }
  21.       
  22.     });
  23.   
  24. });

You could query list items with their attachments via the REST interface as well:

http://YourSharePoint/site/subsite/_api/Web/Lists/GetByTitle('TitleOfYourList')/Items/?$expand=AttachmentFiles

One can use CAML in conjunction with REST by using POST requests and the GetItems method as described here, but as far as I see, the $expand=AttachmentFiles plays not well with the GetItems method. Instead of CAML, you can add the condition, ordering and row limit attributes of your query via $filter, $orderby and $top to the REST query as described here.

After this short overview, let’s try to rewrite our C# code above in JavaScript.

First, I came up with a query like the one below. I requested the very same properties, that we had in the managed client object version: the Title and AttachmentFiles. Note, that I omitted all of the filters I needed in the case of real life list from the CAML query due to the readability as this part is not relevant now in solving the main problem. In contrast to the managed client object code above, I restricted the fields in the CAML query as well via the ViewFields element. That was a mistake, as it turned out later, but I’m a strong believer, that mistakes are there to teach us something new.

  1. var list = web.get_lists().getByTitle("TitleOfYourList");
  2.  
  3. var camlQuery = new SP.CamlQuery();
  4. var query = '<View><ViewFields><FieldRef Name="Title"/><FieldRef Name="AttachmentFiles"/></ViewFields></View>';
  5. camlQuery.set_viewXml(query);
  6.  
  7. var items = list.getItems(camlQuery);
  8. ctx.load(items, 'Include(Title, AttachmentFiles)');

The result was not satisfying, the request failed (screenshot from Fiddler):

image

ErrorMessage: Value does not fall within the expected range.

ErrorTypeName: System.ArgumentException

ErrorStackTrace:

Microsoft.SharePoint.SPFieldMap.GetColumnNumber(String strFieldName, Boolean bThrow)
   at Microsoft.SharePoint.SPListItemCollection.GetColumnNumber(String groupName, Boolean bThrowException)
   at Microsoft.SharePoint.SPListItem.GetValue(SPField fld, Int32 columnNumber, Boolean bRaw, Boolean bThrowException)
   at Microsoft.SharePoint.SPListItem.get_Attachments()
   at Microsoft.SharePoint.SPAttachmentCollection_Client.<GetEnumerator>d__0.MoveNext()
   at Microsoft.SharePoint.Client.ServerStub.<EnumerateChildItems>d__36.MoveNext()
   at Microsoft.SharePoint.Client.ServerStub.WriteChildItems(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJson(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJsonWithMonitoredScope(JsonWriter writer, Object value, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.ServerStub.SPListItemServerStub.WriteOnePropertyValueAsJson(JsonWriter writer, Object target, ClientQueryProperty field, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJson(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteChildItems(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJson(JsonWriter writer, Object obj, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ServerStub.WriteAsJsonWithMonitoredScope(JsonWriter writer, Object value, ClientObjectQuery objectQuery, ProxyContext proxyContext)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.WriteQueryResults(Object obj, ClientObjectQuery objQuery)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessQuery(XmlElement xe)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessOne(XmlElement xe)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.ProcessStatements(XmlNode xe)
   at Microsoft.SharePoint.Client.ClientMethodsProcessor.Process()

You can see the getter method of the Attachments property of the SPListItem object in the fourth line of the stack trace. Let’s have a quick view, how this method works:

  1. public SPAttachmentCollection get_Attachments()
  2. {
  3.    if (this.m_Attachments == null)
  4.    {
  5.        string pbstrUrlPrefix = null;
  6.        object[,] objArrAttachments = null;
  7.        uint pdwRowCount = 0;
  8.        bool bDoQuery = false;
  9.        if (!this.HasExternalDataSource && (this.ID != 0))
  10.        {
  11.            object obj3;
  12.            object obj2 = this.GetValue("Attachments");
  13.            if (obj2 != null)
  14.            {
  15.                bDoQuery = obj2.ToString() != "0";
  16.            }
  17.            SPSecurity.SetListInHttpContext(HttpContext.Current, this.m_Items.List.InternalName);
  18.            SPWeb web = this.Web;
  19.            web.Request.GetAttachmentsInfo(web.Url, this.m_Items.List.InternalName, Convert.ToInt32(this.ID), bDoQuery, out pbstrUrlPrefix, out pdwRowCount, out obj3);
  20.            if (bDoQuery)
  21.            {
  22.                objArrAttachments = (object[,]) obj3;
  23.            }
  24.        }
  25.        this.m_Attachments = new SPAttachmentCollection(this, pbstrUrlPrefix, pdwRowCount, objArrAttachments);
  26.    }
  27.    return this.m_Attachments;
  28. }

As we’ve learnt from Karine’s post, the Attachments field is a flag indicating if a list item has any attachments or not. Obviously, the getter method depends on this property (see the line object obj2 = this.GetValue("Attachments"); in code above), and we should request this property explicitly in our code.

That is the new version of my query:

  1. var camlQuery = new SP.CamlQuery();
  2. var query = '<View><ViewFields><FieldRef Name="Title"/><FieldRef Name="Attachments"/><FieldRef Name="AttachmentFiles"/></ViewFields></View>';
  3. camlQuery.set_viewXml(query);
  4.  
  5. var items = list.getItems(camlQuery);
  6. ctx.load(items, 'Include(Title, Attachments, AttachmentFiles)');

The request was captured by Fiddler as shown bellow. See the properties included in the Include statement, as well the ViewXml:

image

This version works already as expected, but comparing the request with the one captured at C#-based version (see above) I found, that one can omit the Attachments field from the Include statement without any problem.

  1. var camlQuery = new SP.CamlQuery();
  2. var query = '<View><ViewFields><FieldRef Name="Title"/><FieldRef Name="Attachments"/><FieldRef Name="AttachmentFiles"/></ViewFields></View>';
  3. camlQuery.set_viewXml(query);
  4.  
  5. var items = list.getItems(camlQuery);
  6. ctx.load(items, 'Include(Title, AttachmentFiles)');

Furthermore, there is no problem, if you omit the ViewFields element entirely from the CAML query (in our case, we needed later other parts of the CAML query, like the Where element to build up the filters, but it is irrelevant now). That is actually the same query we had with the managed client object model sample at the beginning of this post.

  1. var list = web.get_lists().getByTitle("TitleOfYourList");
  2.  
  3. var camlQuery = new SP.CamlQuery();
  4.  
  5. var items = list.getItems(camlQuery);
  6. ctx.load(items, 'Include(Title, AttachmentFiles)');

Note: Using of the ViewFields element might have an effect on the performance of the query on the server side (especially if you have a lot of columns in your list and you need only a few of them) and so on the response time as well, but after the query has been completed on the server, it seems to be irrelevant regarding to the network bandwidth usage.

That is the full version of the code (to be exact, one of the working ones), including displaying the information (this time in the console only) returned by the server:

  1. var ctx = SP.ClientContext.get_current();
  2. var web = ctx.get_web();
  3.  
  4. var list = web.get_lists().getByTitle("TitleOfYourList");
  5.  
  6. var camlQuery = new SP.CamlQuery();
  7.  
  8. var items = list.getItems(camlQuery);
  9. ctx.load(items, 'Include(Title, Attachments, AttachmentFiles)');
  10.  
  11. ctx.executeQueryAsync(
  12.     function () {
  13.         var enumerator = items.getEnumerator();
  14.         while (enumerator.moveNext()) {
  15.             var item = enumerator.get_current();
  16.             console.log(item.get_item('Title'));
  17.             if (item.get_item('Attachments')) {
  18.                 var attEnumerator = item.get_attachmentFiles().getEnumerator();
  19.                 while (attEnumerator.moveNext()) {
  20.                     var attachment = attEnumerator.get_current();
  21.                     console.log(attachment.get_serverRelativeUrl()); // or get_fileName()
  22.                 }
  23.             }
  24.         }
  25.     },
  26.     function (sender, args) {
  27.         console.log('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
  28.     }
  29. );

That means, we fulfilled our goal by returning the items and their attachments in a single request/response via the JavaScript client object model.

Note, that after the response returned, we first check if there are any attachments for the item by reading the value of item.get_item(‘Attachments’), then accessing the attachments via the item.get_attachmentFiles() method. You could, however, check the existence of the attachments simply by using the get_attachmentFiles() method, without the Attachments field, like this:

var af = item.get_attachmentFiles();
if (af) {
    var attEnumerator = af.getEnumerator();

After extending the CAML query with the filters I need in the Where element, the query returned only the filtered items without any negative effect of the existing functionality, so that part of the task was completed either.

Lessons learned: If you need to read the attachments of your items, you must include the AttachmentFiles field in your Include statement. Including the Attachments field in the Include statement is optional, you need it only if you would like to use it explicitly (see the condition if (item.get_item(‘Attachments’)) in the last code snippet), otherwise you can omit it. If you would like to restrict the fields via the ViewFields element of your CAML query, you should include both of the Attachments and AttachmentFiles fields, but if you don’t restrict the columns by the usage of a ViewFields element, it works as well.

One final note: As so many times before, Fiddler was an invaluable tool to understand how the various versions work, or actually how they don’t work, by comparing the requests sent by the codes down to the wire.

March 23, 2018

How to check if a specific file exists in a folder structure of a SharePoint document library using the client object model

Filed under: CAML, Managed Client OM, SP 2013 — Tags: , , — Peter Holpar @ 22:25

Recently we had to create a utility function that makes it us possible to check if a file having a specific name exists anywhere within a folder structure of a SharePoint document library.

As long as you know not only the title of the document library, but its server relative URL as well, it requires only a single round-trip to the server:

  1. private bool FileExists(ClientContext clientContext, String docLibTitle, String fileName, String rootFolderServerRelativeUrl, String folderPath)
  2. {
  3.     var folderServerRelativeUrl = string.Format("{0}/{1}", rootFolderServerRelativeUrl, folderPath);
  4.     // or use a helper method to combine URL parts
  5.     //var folderServerRelativeUrl = JoinUrlParts(rootFolderServerRelativeUrl, folderPath);
  6.  
  7.     List DocumentsList = clientContext.Web.Lists.GetByTitle(docLibTitle);
  8.  
  9.     CamlQuery camlQuery = new CamlQuery();
  10.     camlQuery.ViewXml = @"<View Scope='Recursive'>
  11.                         <Query>
  12.                             <Where>
  13.                                 <Eq>
  14.                                     <FieldRef Name='FileLeafRef'></FieldRef>
  15.                                     <Value Type='Text'>" + fileName + @"</Value>
  16.                                 </Eq>
  17.                             </Where>
  18.                         </Query>
  19.                 </View>";
  20.     camlQuery.FolderServerRelativeUrl = folderServerRelativeUrl;
  21.     ListItemCollection listItems = DocumentsList.GetItems(camlQuery);
  22.     clientContext.Load(listItems);
  23.     clientContext.ExecuteQuery();
  24.  
  25.     return listItems.Count > 0;
  26. }

Usage:

  1. var webUrl = "http://YourSharePoint/site/subsite&quot;;
  2. string docLibTitle = "Documents";
  3. var rootFolderServerRelativeUrl = "/site/subsite/Shared Documents";
  4. var folderPath = "folder/subfolder";
  5.  
  6. ClientContext clientContext = new ClientContext(webUrl);
  7. var fileName = "document.docx";
  8.  
  9. bool fileFound = FileExists(clientContext, docLibTitle, fileName, rootFolderServerRelativeUrl, folderPath);

Note: If the folderServerRelativeUrl points to a location not within the document library (rootFolderServerRelativeUrl is wrong), the CAML query will ignore the FolderServerRelativeUrl and the entire library will be searched for a matching file. If  the folderPath part is wrong (not existing folder) then no matching item will be found, the query will return always false. Although the SPFolder server-side object model provides an Exists property to check if the folder at the given oath exists, there is no such property for the Folder object in the client object model. As a workaround, you can detect such mistakes by including these two lines of code in the FileExists method before invoking the ExecuteQuery method:

Folder folder = clientContext.Web.GetFolderByServerRelativeUrl(folderServerRelativeUrl);
clientContext.Load(folder);

If either part of the folderServerRelativeUrl is wrong, a File not found exception will be thrown on calling the ExecuteQuery method.

The helper method mentioned in the code is useful if you would not like to bother with leading and trailing slashes in the URL and in the folder path:

  1. public static string JoinUrlParts(params string[] urlParts)
  2. {
  3.     return string.Join("/", urlParts.Where(up => !string.IsNullOrEmpty(up)).ToList().Select(up => up.Trim('/')).ToArray());            
  4. }

If you, however, know only the title of the document library but not its server relative URL you need two round-trips:

  1. private bool FileExists(ClientContext clientContext, String docLibTitle, String fileName, String folderPath)
  2. {
  3.     List docLib = clientContext.Web.Lists.GetByTitle(docLibTitle);
  4.     Folder rootFolder = docLib.RootFolder;
  5.     clientContext.Load(rootFolder, f => f.ServerRelativeUrl);
  6.     clientContext.ExecuteQuery();
  7.  
  8.     string rootFolderServerRelativeUrl = rootFolder.ServerRelativeUrl;
  9.     var folderServerRelativeUrl = string.Format("{0}/{1}", rootFolderServerRelativeUrl, folderPath);
  10.     // or use a helper method to combine URL parts
  11.     //var folderServerRelativeUrl = JoinUrlParts(rootFolderServerRelativeUrl, folderPath);
  12.  
  13.     Folder folder = clientContext.Web.GetFolderByServerRelativeUrl(folderServerRelativeUrl);
  14.     clientContext.Load(folder);
  15.  
  16.     CamlQuery camlQuery = new CamlQuery();
  17.     camlQuery.ViewXml = @"<View Scope='Recursive'>
  18.                         <Query>
  19.                             <Where>
  20.                                 <Eq>
  21.                                     <FieldRef Name='FileLeafRef'></FieldRef>
  22.                                     <Value Type='Text'>" + fileName + @"</Value>
  23.                                 </Eq>
  24.                             </Where>
  25.                         </Query>
  26.                 </View>";
  27.     camlQuery.FolderServerRelativeUrl = folderServerRelativeUrl;
  28.     ListItemCollection listItems = docLib.GetItems(camlQuery);
  29.     clientContext.Load(listItems);
  30.     clientContext.ExecuteQuery();
  31.  
  32.     return listItems.Count > 0;
  33. }

Note: This version already includes the two-liner to check the existence of the folder path. If you don’t need that, remove it.

Usage of this version:

  1. var webUrl = "http://YourSharePoint/site/subsite&quot;;
  2. var docLibTitle = "Documents";
  3. var folderPath = "folder/subfolder";
  4. ClientContext clientContext = new ClientContext(webUrl);
  5. var fileName = "document.docx";
  6.  
  7. bool fileFound = FileExists(clientContext, docLibTitle, fileName, folderPath);

Changing SharePoint Search Preferences from Code

Filed under: PowerShell, Reflection, Search, SP 2013 — Tags: , , , — Peter Holpar @ 22:22

In my recent post I’ve illustrated with C# and PowerShell examples, how to read search preferences info from code, both for the current user as well as for other users. In this post we will see, how to change that preference from code.

We will use the same object, the Microsoft.Office.Server.Search.Administration.UserPreference class. Unlike its static GetUserPreference method, the either static SetUserPreference method has no overload that accepts the SharePoint context (SPContext) as parameter. The single overload of this method accepts a UserPreference instance. That makes our life not easier when it comes later to changing the preferences for another user. Don’t lose the hope, it is not impossible. Bur first things first.

Once we received a UserPreference instance via the GetUserPreference method, you should change certain preference properties, like OpenDocumentsInClient. How to do it? There are two methods, EnableSettings and DisableSetting (both having a parameter of  the nested enumeration type Settings) defined in the UserPreference class. If you would like to activate a setting, you should call the EnableSettings method, if you need to deactivate it, call the DisableSetting method, then finally invoke the SetUserPreference method to persist the changes. For example:

  1. var userPref = UserPreference.GetUserPreference();
  2. // if you would like to open documents in Office client, like Word or Excel
  3. userPref.UpdateSetting(UserPreference.Settings.OpenDocumentsInClient, true);
  4. // if you would like to open documents in Browser (Office Web Apps)
  5. userPref.UpdateSetting(UserPreference.Settings.OpenDocumentsInClient, false);
  6. UserPreference.SetUserPreference(userPref);

Note: This code works only if  you try it within a SharePoint context, like on an application page or in web part. In a console application you will receive an exception when you invoke the overload of the GetUserPreference method without the SPContext parameter:

ArgumentNullException
The value must not be null.
Parameter name: SPContext.Current

As we’ve already seen, there is an overload of the GetUserPreference method that accepts a SPContext parameter, so you could use that to get the preferences, but as there is no such overload for the SetUserPreference method, at least at this points will be the same type of exception thrown again. We will revisit the question shortly, how to set your own preferences from a console application, but we make a quick detour first.

To tell the truth, I don’t like the above pattern at all. Instead of these two methods I created an extension method with a Boolean parameter that encapsulates the functionality:

  1. public static void UpdateSetting(this UserPreference userPreference, UserPreference.Settings setting, bool value)
  2. {
  3.     if (value)
  4.     {
  5.         userPreference.EnableSetting(setting);
  6.     }
  7.     else
  8.     {
  9.         userPreference.DisableSetting(setting);
  10.     }
  11. }

Using this new method one can enable / disable preference settings like:

  1. var userPref = UserPreference.GetUserPreference();
  2. // if you would like to open documents in Office client, like Word or Excel
  3. userPref.EnableSetting(UserPreference.Settings.OpenDocumentsInClient);
  4. // if you would like to open documents in Browser (Office Web Apps)
  5. userPref.DisableSetting(UserPreference.Settings.OpenDocumentsInClient);
  6. UserPreference.SetUserPreference(userPref);

Back to the question, how to set your own preferences when the code runs without SharePoint context, like from a console application?

The “trivial” way is to fake a SharePoint context, using the method described here:

  1. using (SPSite site = new SPSite(url))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         HttpRequest request = new HttpRequest(string.Empty, url, string.Empty);
  6.  
  7.         HttpResponse response = new HttpResponse(new System.IO.StreamWriter(new System.IO.MemoryStream()));
  8.  
  9.         HttpContext ctx = new HttpContext(request, response);
  10.         ctx.Items["HttpHandlerSPWeb"] = web;
  11.         HttpContext.Current = ctx;
  12.  
  13.         Console.WriteLine(SPContext.Current.Web.CurrentUser.LoginName);
  14.  
  15.         var userPref = UserPreference.GetUserPreference();
  16.         userPref.UpdateSetting(UserPreference.Settings.OpenDocumentsInClient, true);
  17.         UserPreference.SetUserPreference(userPref);
  18.  
  19.         //set back the original context (e.g. null)
  20.         HttpContext.Current = null;
  21.     }
  22. }

Another option is, to try to understand, how the SetUserPreference method internally works. It turns out, that it call the internal static UpdatePreference method:

UpdatePreference(preference, false, SPContext.Current);

So I’ve created just another extension method that wraps invoking the UpdatePreference method using Reflection:

  1. public static void Update(this UserPreference userPreference, bool fClearClickHistory, SPContext context)
  2. {
  3.     Type[] paramTypes = { typeof(UserPreference), typeof(bool), typeof(SPContext) };
  4.     MethodInfo updatePreference = userPreference.GetType().GetMethod("UpdatePreference", BindingFlags.Static | BindingFlags.NonPublic, null, paramTypes, null);
  5.     object[] parameters = { userPreference, fClearClickHistory, context };
  6.     updatePreference.Invoke(null, parameters);
  7. }

And a further helper method that accepts a SPContext object as parameter, writes out, preferences of which user we are to change and performs the change itself via the methods we have already:

  1. private void UpdatePreference(SPContext ctx)
  2. {
  3.     var user = ctx.Web.CurrentUser;
  4.     Console.WriteLine("Setting preferences for '{0}'", user.LoginName);
  5.  
  6.     var userPref = UserPreference.GetUserPreference(false, ctx);
  7.     userPref.UpdateSetting(UserPreference.Settings.OpenDocumentsInClient, false);
  8.     userPref.Update(false, ctx);
  9. }

I think the code we achieved using this extension method is much more readable as the former one with the dummy context:

  1. using (SPSite site = new SPSite(url))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         var ctx = SPContext.GetContext(web);
  6.         UpdatePreference(ctx);
  7.     }
  8. }

The code snippets until this point have effect only on the current user. How to change the settings for other user? That is possible either, as soon we combine the methods we already have with impersonation.

First, the version that uses the dummy context:

  1. using (SPSite site = new SPSite(url))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         var user = web.EnsureUser(@"i:0#.w|domain\user");
  6.  
  7.         SPSecurity.RunWithElevatedPrivileges(
  8.           () =>
  9.           {
  10.               using (SPSite impSite = new SPSite(url, user.UserToken))
  11.               using (SPWeb impWeb = impSite.OpenWeb())
  12.               {
  13.                   HttpRequest request = new HttpRequest(string.Empty, url, string.Empty);
  14.  
  15.                   HttpResponse response = new HttpResponse(new System.IO.StreamWriter(new System.IO.MemoryStream()));
  16.  
  17.                   HttpContext impersonatedContext = new HttpContext(request, response);
  18.  
  19.                   impersonatedContext.Items["HttpHandlerSPWeb"] = impWeb;
  20.  
  21.                   HttpContext.Current = impersonatedContext;
  22.  
  23.                   Console.WriteLine(SPContext.Current.Web.CurrentUser.LoginName);
  24.  
  25.                   var userPref = UserPreference.GetUserPreference();
  26.                   userPref.UpdateSetting(UserPreference.Settings.OpenDocumentsInClient, true);
  27.                   UserPreference.SetUserPreference(userPref);
  28.  
  29.                   //set back the original context (e.g. null)
  30.                   HttpContext.Current = null;
  31.               }
  32.           });
  33.     }
  34. }

Next, the other version using Reflection:

  1. using (SPSite site = new SPSite(url))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         var user = web.EnsureUser(@"i:0#.w|domain\user");
  6.  
  7.         SPSecurity.RunWithElevatedPrivileges(
  8.           () =>
  9.           {
  10.               using (SPSite impSite = new SPSite(url, user.UserToken))
  11.               using (SPWeb impWeb = impSite.OpenWeb())
  12.               {
  13.                   var impCtx = SPContext.GetContext(impWeb);
  14.                   UpdatePreference(impCtx);
  15.               }
  16.           });
  17.     }
  18. }

Mission completed.

For those of you who would like to have the same functionality from PowerShell (of course, there are no SharePoint context inherited from the process at all), I include the equivalent methods below.

These are the helper methods we rely on:

  1. function UpdateSetting($userPreference, $setting, $value) {
  2.     If ($value)
  3.     {
  4.         $userPreference.EnableSetting($setting)
  5.     }
  6.     Else
  7.     {
  8.         $userPreference.DisableSetting($setting)
  9.     }
  10. }
  11.  
  12. function Update($userPreference, $fClearClickHistory, $context) {
  13.     $paramTypes = ($up, [bool], [Microsoft.SharePoint.SPContext])
  14.     $updatePreference = $up.GetMethod("UpdatePreference", [System.Reflection.BindingFlags]"Static, NonPublic" , $null, $paramTypes, $null)
  15.     $parameters = ($userPreference, $fClearClickHistory, $context)
  16.     $updatePreference.Invoke($null, $parameters)
  17. }

Furthermore, we declared the following shortcuts:

  1. # shortcut for UserPreference
  2. $up = [Microsoft.Office.Server.Search.Administration.UserPreference]
  3. # shortcut for the nested class Settings in UserPreference
  4. $ups = [Microsoft.Office.Server.Search.Administration.UserPreference+Settings]

Set preferences for the current user via Reflection:

  1. $web = Get-SPWeb $url
  2. $ctx = [Microsoft.SharePoint.SPContext]::GetContext($web)
  3.  
  4. $pref = $up::GetUserPreference($false, $ctx)
  5. UpdateSetting $pref $ups::OpenDocumentsInClient $true
  6. Update $pref $false $ctx

Set preferences for another user via Reflection:

  1. $userName = 'i:0#.w|domain\user'
  2.  
  3. $web = Get-SPWeb $url
  4. $user = $web.EnsureUser($userName)
  5. $userToken = $user.UserToken
  6.  
  7. $impersonatedSite = New-Object Microsoft.SharePoint.SPSite($url, $userToken)
  8. $ctx = [Microsoft.SharePoint.SPContext]::GetContext($impersonatedSite.RootWeb)
  9.  
  10. $pref = $up::GetUserPreference($false, $ctx)
  11. UpdateSetting $pref $ups::OpenDocumentsInClient $true
  12. Update $pref $false $ctx

Set preferences for the current user using a dummy context (see this post about injecting a fake SharePoint context into PowerShell):

  1. $web = Get-SPWeb $url
  2. $ctx = [Microsoft.SharePoint.SPContext]::GetContext($web)
  3.  
  4. $sw = New-Object System.IO.StringWriter
  5. $request = New-Object System.Web.HttpRequest "", $url, ""
  6. $response = New-Object System.Web.HttpResponse $sw
  7. $dummyContext = New-Object System.Web.HttpContext $request, $response
  8. [System.Web.HttpContext]::Current = $dummyContext
  9. $dummyContext.Items["HttpHandlerSPWeb"] = $ctx.Web;
  10.  
  11. $pref = $up::GetUserPreference($false, $ctx)
  12. #$pref.EnableSetting($ups::OpenDocumentsInClient)
  13. #or
  14. #$pref.DisableSetting($ups::OpenDocumentsInClient)
  15. UpdateSetting $pref $ups::OpenDocumentsInClient $true
  16. $up::SetUserPreference($pref)
  17.  
  18. [System.Web.HttpContext]::Current = $null

Set preferences for another user using a dummy context:

  1. $userName = 'i:0#.w|domain\user'
  2.  
  3. $web = Get-SPWeb $url
  4. $user = $web.EnsureUser($userName)
  5. $userToken = $user.UserToken
  6.  
  7. $impersonatedSite = New-Object Microsoft.SharePoint.SPSite($url, $userToken)
  8. $ctx = [Microsoft.SharePoint.SPContext]::GetContext($impersonatedSite.RootWeb)
  9.  
  10. $sw = New-Object System.IO.StringWriter
  11. $request = New-Object System.Web.HttpRequest "", $url, ""
  12. $response = New-Object System.Web.HttpResponse $sw
  13. $dummyContext = New-Object System.Web.HttpContext $request, $response
  14. [System.Web.HttpContext]::Current = $dummyContext
  15. $dummyContext.Items["HttpHandlerSPWeb"] = $ctx.Web;
  16.  
  17. $pref = $up::GetUserPreference($false, $ctx)
  18. #$pref.EnableSetting($ups::OpenDocumentsInClient)
  19. #or
  20. #$pref.DisableSetting($ups::OpenDocumentsInClient)
  21. UpdateSetting $pref $ups::OpenDocumentsInClient $true
  22. $up::SetUserPreference($pref)
  23.  
  24. [System.Web.HttpContext]::Current = $null

Querying SharePoint Search Preferences from Code

Filed under: PowerShell, Search, SP 2013 — Tags: , , — Peter Holpar @ 22:20

Recently a user complained, that he is not able to open office documents from the SharePoint portal of the company when working at home. He attached a screenshot of the error message in the browser to his mail. From this screenshot was it obvious, that there is an issue with the accessibility of the Office Web Applications server (OWA, also known as WAC), see the word wac in the address on the browser screen or WopiFrame.aspx in the address bar.

image

As it turned out, the WAC-server of the company has not been published externally via the firewall, but it was on purpose. Users should have been able to work with documents using their locally installed Office applications.

As you might know, you can configure the behavior, if document would be opened in browser or in the Office client application instead on either the site collection or on the document library level (see details here). The documents the user complained about were located in a library with the setting “Use the server default (Open in the client application)”, but it has not helped, when we changed it to “Open in the client application” explicitly.

It was really curious, but after a little while it turned out, that he wanted to open the document not from the library, but from a search result. At least a step further to the solution, have we thought.

You should know, that the behavior, if the Office documents get opened in the client application or in the browser is independent from the site collection level settings as well as from the document library settings. There is a (in my personal opinion pretty hidden) Preference link at the bottom of the search results page:

image

On this page the users can configure their own preferences, among others, if they would like to open the Office documents in the client application or in the browser:

image

It’s a cool option to enable users to decide which way they prefer, although it is pretty inconsistent with the other options (available for the administrators) we mentioned earlier. But there is an even bigger issue (at least, for me) with that. There is (as far as I know) no option / UI for administrators to query the value configured for a user, not to mention, how to change it remotely, without end user interaction.

Although it might have been the easiest choice to ask the user, which value he has configured for himself , I’m not the man of easy options if there might be a programmatic approach as well and a chance to learn something new. So let’s see, what I’ve learned.

The user preferences regarding the search are available via the Microsoft.Office.Server.Search.Administration.UserPreference class. If you need the user preferences from the current SPContext (e.g. for the current user), you can use either the static GetUserPreference() method or the other static overload GetUserPreference(bool lookupFromCache). If, however, you need the preference for another user, you can inject it via the static GetUserPreference(bool lookupFromCache, SPContext context) method.

For example, the DisplayPreference method below displays a few of the available preferences from a context it receives as parameter:

  1. private void DisplayPreference(SPContext ctx)
  2. {
  3.     var user = ctx.Web.CurrentUser;
  4.     Console.WriteLine("Reading preferences for '{0}'", user.LoginName);
  5.  
  6.     var userPref = UserPreference.GetUserPreference(false, ctx);
  7.  
  8.     Console.WriteLine("ShowPrequerySuggestion: {0}", userPref.IsSettingEnabled(UserPreference.Settings.ShowPrequerySuggestion));
  9.     Console.WriteLine("ShowPersonalSuggestions: {0}", userPref.IsSettingEnabled(UserPreference.Settings.ShowPersonalSuggestions));
  10.     Console.WriteLine("OpenDocumentsInClient: {0}", userPref.IsSettingEnabled(UserPreference.Settings.OpenDocumentsInClient));
  11. }

The following code snippet (taken from a console application) invokes the DisplayPreference method first to display the preferences of the current user, then again to display the preferences of an impersonated user:

  1. using (SPSite site = new SPSite(url))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         var ctx = SPContext.GetContext(web);
  6.         DisplayPreference(ctx);
  7.  
  8.         var user = web.EnsureUser(@"i:0#.w|domain\user");
  9.  
  10.         SPSecurity.RunWithElevatedPrivileges(
  11.           () =>
  12.           {
  13.               using (SPSite impSite = new SPSite(url, user.UserToken))
  14.               using (SPWeb impWeb = impSite.OpenWeb())
  15.               {
  16.                   var impCtx = SPContext.GetContext(impWeb);
  17.  
  18.                   DisplayPreference(impCtx);
  19.               }
  20.           });
  21.     }
  22. }

Of course, if your code runs in a SharePoint process, you can get the context as SPContext.Current as well for the current user.

The same information is available via PowerShell either. For example, displaying preferences for the current user:

  1. $web = Get-SPWeb $url
  2. $ctx = [Microsoft.SharePoint.SPContext]::GetContext($web)
  3.  
  4. # shortcut for UserPreference
  5. $up = [Microsoft.Office.Server.Search.Administration.UserPreference]
  6. # shortcut for the nested class Settings in UserPreference
  7. $ups = [Microsoft.Office.Server.Search.Administration.UserPreference+Settings]
  8.  
  9. $pref = $up::GetUserPreference($false, $ctx)
  10. $pref.IsSettingEnabled($ups::ShowPrequerySuggestion)
  11. $pref.IsSettingEnabled($ups::ShowPersonalSuggestions)
  12. $pref.IsSettingEnabled($ups::OpenDocumentsInClient)

If you need the preferences of another user, you should impersonate it first as described here. After the impersonation, the code is pretty the same as earlier:

  1. $userName = 'i:0#.w|domain\user'
  2.  
  3. $web = Get-SPWeb $url
  4. $user = $web.EnsureUser($userName)
  5. $userToken = $user.UserToken
  6.  
  7. $impersonatedSite = New-Object Microsoft.SharePoint.SPSite($url, $userToken)
  8. $ctx = [Microsoft.SharePoint.SPContext]::GetContext($impersonatedSite.RootWeb)
  9.  
  10. # shortcut for UserPreference
  11. $up = [Microsoft.Office.Server.Search.Administration.UserPreference]
  12. # shortcut for the nested class Settings in UserPreference
  13. $ups = [Microsoft.Office.Server.Search.Administration.UserPreference+Settings]
  14.  
  15. $pref = $up::GetUserPreference($false, $ctx)
  16. $pref.IsSettingEnabled($ups::ShowPrequerySuggestion)
  17. $pref.IsSettingEnabled($ups::ShowPersonalSuggestions)
  18. $pref.IsSettingEnabled($ups::OpenDocumentsInClient)

Using this code we were able to detected that the complaining user has really the wrong preference (OpenDocumentsInClient was false). Now we had two choices: either to call the user, and ask him to change the preference, or to find a solution, how it would be possible to change it from code on behalf of the user remotely. Of course, this time we didn’t want to change the preferences without the explicit permission of the user, so took option 1, but I show you in my next post, how you could do it from code.

Older Posts »

Create a free website or blog at WordPress.com.