Performance testing SharePoint applications

Note (to avoid disappointment): This post is not intended to be a general introduction or walkthrough of performance testing of web / SharePoint applications. It is rather just a quick tip on a tool I found useful performing such tasks.

A developer should never forget, that functional testing is only one – although a very important one – part of the application testing. There are other things to test, for example, how do the components of your system behave in a stress situation.

Recently we had to perform a load test on a server-intensive Silverlight application. The client side calls a lot of RIA web services methods to query and update data stored in the SQL database and other back-end systems. So I had to choose a tool that makes our test easy to perform.

In the past we already had a similar issue with a SharePoint application, where we decided to use Web Performance Test Recorder included in Visual Studio Ultimate for the load test, but unfortunately in this case the Ultimate edition was no option due to licensing considerations.

My next idea was Fiddler, the free tool I used quite frequently to monitor network traffic (especially when playing with the SharePoint Client Object Model), and whose ability to record and perform such tests I was beware of, but before starting the actual test I wanted to check the SL forums to see the experience of other developers in this area. In this thread I found a particularly interesting Fiddler extension called StresStimulus, as well as a quick start about how to use this tool.

I’ve downloaded a trial version of this product, and found it can really help the web developer (on a relative low budget) to monitor the performance of the network and the server on a high load situation.

A few of the nice features of the tool are the ability to parameterize the requests sent to the server as well as the capability to use a set of username/password pairs for authentication.

// Regarding SharePoint, the TechNet article about performance testing describes the Visual Studio Team System (Team Test Load Agent), and MSDN shows the usage of the Web Performance Test Recorder, but the list of performance counters to monitor is independent from the tool you choose for testing.

The tale of the junked MVP award notification

Today morning I received a mail that described some steps I should complete as a new MVP. First I thought it’s only a silly joke of one of my friends. As far as I know MVP is awarded / renewed on the first day of each quarter and I received no such mail on last Friday. To tell the truth, I did not really expect such award this time.

After a short period of time, clicking the links in the mail I started only to believe that I might have really been awarded MVP for SharePoint Server. The final proof was the notification mail from last Friday that I dug out from the Junk E-Mail folder in Outlook. Really bizarre.

Although I work with SharePoint since the beta version of SPS 2001 (a.k.a Tahoe), I feel it all began in 2006 when my group started to work with MOSS 2007, and due to the undocumented nature of the product we got a lot of help from the blogosphere and the international SharePoint community. Of course we had our own experiences and I felt I have to share this knowledge with others in the same boat: I’ve started to answer questions on forums and write my own blog posts.

It seems I’m a MVP for SharePoint Server now, so it’s time to say a big thank you a lot of people.

First and foremost I would like to thank the inspiration of (now) fellow MVPs and all other SharePoint bloggers and conference speakers.

I’m grateful for my colleagues (most of them are now unfortunately ex-colleague) I worked together on several SharePoint projects. We learned a lot from each other, and not only about SharePoint.

I’m much obligated the guys at MS to make me possible to share my experience to the Hungarian SharePoint at developer community events and for their support in nomination.

Of course, I’m thankful for forum members and blog readers to motivate me through your questions and feedback.

Last but definitely not least my family has a great part in this award. They had to miss their husband / father / son a lot of time when he spent his not too much spare time with his favorite SharePoint research.

Thank you all, I promise to try to do my best in the future as well to deserve your trust!

SharePoint menus lost their functionality

In the last few days I was developing a SharePoint 2010 solution, including a feature with a custom action.

In the middle of the testing / fine tuning process I suddenly realized that my custom action does not work anymore. When I clicked on the menu simple nothing happened.

Restarting the browser (Internet Explorer 8 in my case) or clearing IE cache to remove potentially corrupted cached .js files does not help nor does the “standard” IISRESET and server restart to exclude server side issues.

JavaScript was enabled, other web pages worked as expected, even the ribbon controls on my SharePoint pages worked like before.

However, built-in menus in the edit control block (ECB), Site Actions and Welcome Menu also suffered from the same problem.

To decide whether it is a client or server side issue, I visited another server from my machine. This server runs MOSS 2007 but I found the same problem with the menu when visiting from my developer server.

It gave me some hope and help that when started my browser InPrivate or started its x64 version the sites showed their normal behavior.

 

At this point I started suspecting for JavaScript interference with installed IE add-ons. To start Internet Explorer with all add-ons disabled, you can use the –extoff command prompt parameter. See Internet Explorer Command-Line Options for details. The image below shows Internet Explorer started in Add-ons disabled mode.

I found that disabling the add-ons solved the issue, so the next step was to locate the add-on that was responsible for the behavior. I disabled the add-ons one by one in the Tools / Manage Add-ons menu in IE 8.

In my case a DivX add-on was accountable for the issue. A few days ago I installed a DivX codec that included add-ons as well, but the behavior started only yesterday, probably when I restart IE the first time after the codec installation.

Declaratively adding a lookup field to a list schema using the name of the referenced list

This topic is really not a new one. I read already debates several times about if it is possible or not, see for example this post in Josh Gaffey’s blog:

Add SharePoint lookup column declaratively through CAML XML

or this post from Chris O’Brien’s blog:

Creating lookup columns as a feature

There are a lot of comments pro and cons in these posts.

Since I’ve got this question again on the MSDN forum (see create lookup column in list template without guid of parent list), I decided to check the issue myself.

In the past I deployed lists with lookup fields using the name of the referenced list formerly using a custom deployment tool, now I had to create a declarative feature using CAML definitions.

In my test I was to create a lookup field to the Title field of the standard Tasks list.

Based on the guide of Josh in the above post I first try to create a lookup site column.

To achieve this, I’ve created a simple feature that included the following fields.xml file:

1. <?xml version="1.0" encoding="utf-8"?>
2. <Elements xmlns="http://schemas.microsoft.com/sharepoint/">
3.     <Field ID="{95d89725-eb97-428b-bc79-ee02ca8b724c}"
4.         Name="TestField"
5.         SourceID="http://schemas.microsoft.com/sharepoint/v3"
6.         StaticName="TestField"
7.         Group="Test site columns"
8.         Type="Lookup"
9.         DisplayName="Test field"
10.         List="Lists/Tasks"
11.         FieldRef="ID"
12.         ShowField="Title" />
13. </Elements>

The feature.xml file looks like this:

1. <?xml version="1.0" encoding="utf-8"?>
2.   <FeatureId="2AABD552-10B1-4561-8A19-2311D39C9A2E"
3.           Title="List name based lookup field demo feature (site column)"
4.           Description="The sole purpose of this feature is to demonstrate how to create list name based lookup field in WSS 3.0 using a site column"
5.           Version="1.0.0.0"
6.           Hidden="FALSE"
7.           Scope="Site"
8.           xmlns="http://schemas.microsoft.com/sharepoint/">
9.   <ElementManifests>
10.     <ElementManifest Location="fields.xml"/>
11.   </ElementManifests>
12. </Feature>

Activating the feature resulted in the following in the site columns list:

At first sight that was OK, but after checking the details it turned out to be not perfect:

As you can see on the image above, the place of the parent list was empty.

Checking the field from code showed that the LookupList property of the site column was “Lists/Tasks”, so it was not resolved to the list GUID on feature activation.

When I tried to add the site column to a list, I’ve received the following exception:

Exception from HRESULT: 0x80040E07   at Microsoft.SharePoint.Library.SPRequestInternalClass.AddField(String bstrUrl, String bstrListName, String bstrSchemaXml, Int32 grfAdd)

  at Microsoft.SharePoint.Library.SPRequest.AddField(String bstrUrl, String bstrListName, String bstrSchemaXml, Int32 grfAdd)

I’ve also tried to create a custom list definition in the same feature, but when I tried to create a new instance based on that definition, I received the following exception:

Cannot complete this action.
Please try again.   at Microsoft.SharePoint.Library.SPRequestInternalClass.CreateListFromFormPost(String bstrUrl, String& pbstrGuid, String& pbstrNextUrl)
   at Microsoft.SharePoint.Library.SPRequest.CreateListFromFormPost(String bstrUrl, String& pbstrGuid, String& pbstrNextUrl)

Although some comments suggested to remove the ShowField attribute to get it work, in my experience it did not help. It only made the case even worse, as now the column information was also empty (as one could expect logically):

So I have to find an alternative way. I decided to try it with a custom list definition that contains the field definition itself.

If you are new to creating custom list definitions, you can read more about that here:

How to: Create a Custom List Definition

I’ve created a feature that contains the custom list definition. The main modifications of the simple custom list schema are the followings:

1. <ContentTypes>
2.     <ContentTypeID="0×01"
3.     Name="Item"
4.     Group="Item group"
5.     Description="Item description"
6.     Version="0">
7.     <FieldRefs>
8.       <FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" Required="TRUE" ShowInNewForm="TRUE" ShowInEditForm="TRUE"/>
9.       <!– Title –>
10.       <FieldRef ID="{95d89725-eb97-428b-bc79-ee02ca8b7225}" Name="TestField" Required="FALSE" ShowInNewForm="TRUE" ShowInEditForm="TRUE"/>
11.       <!– Test field –>
12.     </FieldRefs>
13.     <XmlDocuments>
14.       <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">
15.         <FormTemplates xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">
16.           <Display>ListForm</Display>
17.           <Edit>ListForm</Edit>
18.           <New>ListForm</New>
19.         </FormTemplates>
20.       </XmlDocument>
21.     </XmlDocuments>
22.   </ContentType>
23.   <ContentTypeRef ID="0×0120" />
24. </ContentTypes>
25. <Fields>
26.   <Field Type="Lookup"
27.          DisplayName="Test field"
28.          Required="FALSE"
29.          List="Lists/Tasks"
30.          ShowField="Title"
31.          UnlimitedLengthInDocumentLibrary="FALSE"
32.          ID="{95d89725-eb97-428b-bc79-ee02ca8b7225}"
33.          SourceID="http://schemas.microsoft.com/sharepoint/v3"
34.          StaticName="TestField"
35.          Name="TestField" />
36. </Fields>

and a reference for the lookup field was added to the ViewFields:

1. <ViewFields>
2.   <FieldRef Name="Attachments">
3.   </FieldRef>
4.   <FieldRef Name="LinkTitle">
5.   </FieldRef>
6.   <FieldRef Name="TestField">
7.   </FieldRef>
8. </ViewFields>

Next, I’ve installed and activated the feature:

stsadm -o installfeature -name ListNameBasedLookUp
stsadm -o activatefeature -name ListNameBasedLookUp -url http://yoursite

The custom list definition appeared on the available list types:

I’ve created a new instance called LookUpList1:

The new list instance contained the lookup field:

And the definition of the field was correct. It pointed to the right list (Tasks) and to the right field (Title):

The field was added to the default view as expected:

In the meantime I’ve created to item in the Tasks list:

When I created a new item, the values of the lookup field came from the specified source list and field:

And last but not least, the item is saved correctly to the list:

So my experience matches to Scott’s comment in Josh’s post:

“To make this method work, the field definition must exist in a list definition since the wiring up to the GUID happens when you the provision the list, rather than when you add a column to an existing content type/list.”

At the end some note:

If you want to alter the feature XML files and would like to avoid caching issues, I suggest you to alter the feature and field GUIDs each time you deploy (deactivate/uninstall/alter/install/activate/IISRESET) the feature. Ignoring this hint may cause you a lot of headache. Believe me, I tell you that from my own experience!

Another side note, that the experience may depend on the version number of SharePoint in my environment I tested it with the December 2009 Cumulative Update (12.0.0.6524).

You can find the full sample feature here.

Multivalue AutoComplete WinForms TextBox for tagging

In one of my recent work I created a WinForms application that interacts with WSS 3.0 through web services. On the server side I created some kind of tagging with built-in WSS fields, but I had to solve the tagging in the client application as well.

The TextBox control has the necessary properties to create a single value autocomplete control, see the AutoCompleteMode and related properties.

You can find a sample for their usage here:

• Creating winforms AutoComplete TextBox using C# in Windows application

There are several implementations for custom autocomplete textboxes on the web, but none of these I’ve tried met my plans. I was to create something similar to implementation as the Windows Live Writer handles tags for blog posts.

Finally I’ve decided to create my own control. To tell the truth, I think it took less time than I’ve spent for looking up an existing solution previously.

In this post I present the results and some lessons I’ve learnt on the job.

I’ve implemented my control derived from the TextBox control. It has a private ListBox control to handle suggestions.

1. private void ShowListBox()
2. {
3.     if (!_isAdded)
4.     {
5.         Parent.Controls.Add(_listBox);
6.         _listBox.Left = this.Left;
7.         _listBox.Top = this.Top + this.Height;
8.         _isAdded = true;
9.     }
10.     _listBox.Visible = true;
11. }
12.  
13. private void ResetListBox()
14. {
15.     _listBox.Visible = false;
16. }

I had to handle the KeyUp event to update the ListBox to show the matching items, and the KeyDown event to handle special keys. These special keys are Up and Down to navigate between suggestions and the Tab to accept suggestions.

1. private void this_KeyUp(object sender, KeyEventArgs e)
2. {
3.     UpdateListBox();
4. }
5.  
6. private void this_KeyDown(object sender, KeyEventArgs e)
7. {
8.     switch (e.KeyCode)
9.     {
10.         case Keys.Tab:
11.             {
12.                 if (_listBox.Visible)
13.                 {
14.                     InsertWord((String)_listBox.SelectedItem);
15.                     ResetListBox();
16.                     _formerValue = this.Text;
17.                 }
18.                 break;
19.             }
20.         case Keys.Down:
21.             {
22.                 if ((_listBox.Visible) && (_listBox.SelectedIndex < _listBox.Items.Count – 1))
23.                 {
24.                     _listBox.SelectedIndex++;
25.                 }
26.                 break;
27.             }
28.         case Keys.Up:
29.             {
30.                 if ((_listBox.Visible) && (_listBox.SelectedIndex > 0))
31.                 {
32.                     // it is a double – for the case the blog engine removes on of them
33.                     _listBox.SelectedIndex–;
34.                 }
35.                 break;
36.             }
37.     }
38. }

Since by default Tab is to change the focus and the control that accepts user input, we had to override the IsInputKey method.

1. protected override bool IsInputKey(Keys keyData)
2. {
3.     switch (keyData)
4.     {
5.         case Keys.Tab:
6.             return true;
7.         default:
8.             return base.IsInputKey(keyData);
9.     }
10. }

In this implementation I use the semicolon character (‘;’) as a value separator, but it is easy to extend the solution to a configurable separator.

The UpdateListBox method updates the suggestion based on matches for the word start. It only suggests values not already selected. The GetWord method gets the “word” (the text between separator character or begin  / end of the text) we are currently typing. InsertWord inserts the accepted suggestion into the place we are typing in.

1. private void UpdateListBox()
2. {
3.     if (this.Text != _formerValue)
4.     {
5.         _formerValue = this.Text;
6.         String word = GetWord();
7.  
8.         if (word.Length > 0)
9.         {
10.             String[] matches = Array.FindAll(_values,
11.                 x => (x.StartsWith(word) && !SelectedValues.Contains(x)));
12.             if (matches.Length > 0)
13.             {
14.                 ShowListBox();
15.                 _listBox.Items.Clear();
16.                 Array.ForEach(matches, x => _listBox.Items.Add(x));
17.                 _listBox.SelectedIndex = 0;
18.                 _listBox.Height = 0;
19.                 _listBox.Width = 0;
20.                 this.Focus();
21.                 using (Graphics graphics = _listBox.CreateGraphics())
22.                 {
23.                     for (int i = 0; i < _listBox.Items.Count; i++)
24.                     {
25.                         _listBox.Height += _listBox.GetItemHeight(i);
26.                         // it item width is larger than the current one
27.                         // set it to the new max item width
28.                         // GetItemRectangle does not work for me
29.                         // we add a little extra space by using '_'
30.                         int itemWidth = (int)graphics.MeasureString(((String)_listBox.Items[i]) + "_", _listBox.Font).Width;
31.                         _listBox.Width = (_listBox.Width < itemWidth) ? itemWidth : _listBox.Width;
32.                     }
33.                 }
34.             }
35.             else
36.             {
37.                 ResetListBox();
38.             }
39.         }
40.         else
41.         {
42.             ResetListBox();
43.         }
44.     }
45. }
46.  
47. private String GetWord()
48. {
49.     String text = this.Text;
50.     int pos = this.SelectionStart;
51.  
52.     int posStart = text.LastIndexOf(';', (pos < 1) ? 0 : pos – 1);
53.     posStart = (posStart == -1) ? 0 : posStart + 1;
54.     int posEnd = text.IndexOf(';', pos);
55.     posEnd = (posEnd == -1) ? text.Length : posEnd;
56.  
57.     int length = ((posEnd – posStart) < 0) ? 0 : posEnd – posStart;
58.  
59.     return text.Substring(posStart, length);
60. }
61.  
62. private void InsertWord(String newTag)
63. {
64.     String text = this.Text;
65.     int pos = this.SelectionStart;
66.     
67.     int posStart = text.LastIndexOf(';', (pos < 1) ? 0 : pos – 1);
68.     posStart = (posStart == -1) ? 0 : posStart + 1;
69.     int posEnd = text.IndexOf(';', pos);
70.  
71.     String firstPart = text.Substring(0, posStart) + newTag;
72.     String updatedText = firstPart + ((posEnd == -1) ? "" : text.Substring(posEnd, text.Length – posEnd));
73.  
74.  
75.     this.Text = updatedText;
76.     this.SelectionStart = firstPart.Length;
77. }

The Values and SelectedValues properties are to interact with the host application. Values accepts the possible suggestions, SelectedValues returns the tags applied.

1. public String[] Values
2. {
3.     get
4.     {
5.         return _values;
6.     }
7.     set
8.     {
9.         _values = value;
10.     }
11. }
12.  
13. public List<String> SelectedValues
14. {
15.     get
16.     {
17.         String[] result = Text.Split(new char[] { ';' }, StringSplitOptions.RemoveEmptyEntries);
18.         return new List<String>(result);
19.     }            
20. }

The following code shows a simple host application:

1. using System;
2. using System.Collections.Generic;
3. using System.Windows.Forms;
4.  
5. namespace AutoComplete
6. {
7.     public partial class TestForm : Form
8.     {
9.         private String[] _values = { "one", "two", "three", "tree", "four" };
10.  
11.         public TestForm()
12.         {
13.             InitializeComponent();
14.             // AutoComplete is our special textbox control on the form
15.             AutoComplete.Values = _values;
16.         }
17.  
18.         // ShowSelection is a button to populate
19.         // the SelectedList listbox with selected values
20.         private void ShowSelection_Click(object sender, EventArgs e)
21.         {
22.             SelectedList.Items.Clear();
23.             List<String> selectedValues = AutoComplete.SelectedValues;
24.             Array.ForEach(selectedValues.ToArray(), selectedValue => SelectedList.Items.Add(selectedValue.Trim()));
25.         }
26.     }
27. }

The following figures show the application in action.

On the first one you can see that the list box is cropped by the application borders.

It is a known issue. I’ve experienced with the ToolStripDropDown as described in this thread:

Make user control display outside of form boundry

I’ve found, that in this case the control is really not cropped, but I had issues by handling key events, for example, when I typed further it has no effect to the content of the textbox. Since I felt it is more serious that the cropping effect, I decided to keep with the simpler original solution.

The next figure illustrate that the suggestion list contains only the items (tags) not already selected. In this case the suggestion three is missing from the list.

When pressing the >> button, the selected values from the textbox control are transferred to the ListBox of the host application.

You can find the full source of the control and the host application on CodePlex.

Multiple column lookup field for SharePoint

I think I’m not alone missing the possibility in WSS 3.0 to include multiple columns from the target list when creating a lookup or user field in a list.

If you have read my former posts about custom fields and reordering of list fields you already know what you need to create such field types yourself.

In this post I won’t include lengthy code snippets, instead I’ve created a CodePlex project for the source code.

To illustrate the working of the fields I’ve created a custom list called Companies in SharePoint using the following fields:

I’ve added the following items to the list:

Next I’ve created a list called Contracts, and added my custom multi column lookup field (let’s call it master field) as shown on the following figure:

You might notice that the user interface of the field settings is not very sophisticated. You are absolutely right, it is not very error prone to specify the internal names of the related fields (the ones you would like to include into the list additionally to the original lookup field) using a text field separated by a pipe character (‘|’). In this implementation I simply inherited the original FieldEditorUserControl of the lookup field. In forthcoming releases I might have more time to create an advanced one, but in the current phase it was not the most important part of the development.

After adding your new field you can see that only the master field is visible on the Columns section.

But when you check the field order page, you can see that there are additional columns for your related fields as well. It is important that our master field is ordered automatically to the last position, as it must be able to interact with related fields on edit and new list forms.

Furthermore, if you add the master field to the default view on creation, all related fields are automatically added as well.

In the next step I create a new item. Notice, that only the master field is visible on the form.

After adding some additional items I had the following content in the Contracts list:

You can see that related fields are filled automatically.

If you go back to the field settings page of the master field, you can modify the value of the related fields (add or remove fields). Values in columns for added related fields are automatically populated as well, however I should note that this pattern might be not ideal if you have a lot of items in your list or there are multiple parallel editors and enabled check-outs, as it can cause performance issues or conflicts due to checked-out items.

There is an additional field that is very similar but it is for user field.

You can add the custom field to you list as shown here:

And here is a list item having the custom field:

 

Here are the internal name – display name pairs of some common user fields you might be interested in:

Name – Account
EMail – Work e-mail
Department – Department
Title – Name
FirstName – First name
LastName – Last name
WorkPhone – Work phone
Office – Office
UserName – User name

If you would like to enable multiple values for the field, it is important to install the following update on the server:

Description of the Windows SharePoint Services 3.0 Cumulative Update Server hotfix package (Sts-x-none.msp): December 15, 2009

It includes the fix for this issue:

"You develop a custom field type that inherits from the SPFieldLookUp class. You want to store multiple values in a field of that custom field type. Therefore, you set the AllowMultipleValues property to True. However, after you set the AllowMultipleValues property to True, the field type is displayed as Lookup instead of the custom field type."

My event receiver doesn’t fire

On MSDN forums it’s a frequent complaint that somebody creates an event receiver but it does not fire. Based on my experience there is a few main types of this situation. In this post I try to help you to identify the issue and help to avoid or correct the problem.

The first question is why do you think the receiver does not fire? You might expect the receiver to do something (like create or delete a list item, send a mail, etc.), but it does not. Most time there is some kind of exception or a mistake in a condition that cause the code to do something different than you would like it to do.

But it does not mean automatically the receiver is not fired. So the first main type of “does not fire” is that where the receiver fires, but fails to do the expected activity, and it leads you to the incorrect assumption that it does not fire.

You can avoid this mistake by following the suggestions below.

Always use a try / catch / finally block within you receiver, and trace out the exceptions if happen. Also, it is good practice to trace out each entry and exit points in you receiver.

You should not reinvent the wheel so don’t want to log the trace information using your own methods, like simple opening a text file for write access and append the log info to the file. This can easily lead to file lock issues between receiver threads.

Instead, use the built-in .NET tracing infrastructure, like the Trace class from System.Diagnostics namespace namespace. You can use its methods (for example, TraceInformation) to create output from your event receiver.

1. using System;
2. using System.Web;
3. using Microsoft.SharePoint;
4. using System.Diagnostics;
5.  
6. namespace YourReceivers
7. {
8.     public class Receiver : SPItemEventReceiver
9.     {
10.         public override void ItemUpdated(SPItemEventProperties properties)
11.         {
12.             try
13.             {
14.                 Trace.TraceInformation("YourReceivers.Receiver ItemUpdated starting…");
15.                 // do the job here
16.             }
17.             catch (Exception ex)
18.             {
19.                 Trace.TraceError("YourReceivers.Receiver ItemUpdated exception: {0}", ex.Message);
20.             }
21.             finally
22.             {
23.                 Trace.TraceInformation("YourReceivers.Receiver ItemUpdated finished");
24.             }
25.         }
26.     }
27. }

Trace every method entry and exit points as well as result of conditions.

To catch the output you can the use the WinDbg tool, or configure your web application to redirect trace output to a file (see <listeners> Element for <trace>  and the concepts of Trace Listeners). This way you get feedback if your receiver is called at all, and what happens under the hood without debugging, so this technique is useful either on production environment where you cannot use Visual Studio to debug the code.

You can even write your log messages to SharePoint Unified Logging System as described here:

• Logging data to the 12 hive log (using the LogString and DebugLogString methods of the  PortalLog class in the Microsoft.Office.Server.Diagnostics namespace)
• SharePoint: Writing messages to ULS (Unified Logging System)

The following post shows how to create a custom trace listener that writes to the ULS logs:

• Writing a custom trace listener to log to the 12 hive log from the Enterprise Library logging block

Of course, on the development environment you can get the most information by setting a breakpoint at the entry of the event receiver method and attaching the debugger to the worker process, so you can catch the event if it fired. One of my former posts shows you how to attach the debugger easier.

Second main group of “does not fire” issues is where the receiver is not registered correctly (for example, typo in class name or public key token), not registered at all, or the class you register cannot be loaded or does not implements the required methods or these methods are not public.

There are several ways that can be used to register or check the registration of an event receiver. For example, you can do that from your custom code (e.g. command line utility using the SPList.EventReceivers property,  and SPEventReceiverDefinition class) or from a 3rd party tool, like Patrick Tisseghem’s Event Handler Explorer.

The next code shows an example how to create a console application that checks the registered receiver classes and verifies the assemblies using Reflection:

1. using System;
2. using System.Collections.Generic;
3. using System.Reflection;
4. using System.IO;
5. using Microsoft.SharePoint;
6. using Microsoft.SharePoint.Utilities;
7.  
8. namespace ReceiverCheck
9. {
10.     class Program
11.     {
12.         static void Main(string[] args)
13.         {
14.             if (args.Length == 1)
15.             {
16.                 try
17.                 {
18.                     String url = args[0];
19.                     using (SPSite site = new SPSite(url))
20.                     {
21.                         using (SPWeb web = site.OpenWeb())
22.                         {
23.                             Program prog = new Program();
24.                             prog.TestReceivers(web);
25.                         }
26.                     }
27.                 }
28.                 catch (Exception ex)
29.                 {
30.                     Console.WriteLine("ERROR: {0}", ex.Message);
31.                 }
32.             }
33.             else
34.             {
35.                 Console.WriteLine("Usage: ReceiverCheck [URL of SharePoint site]");
36.             }
37.         }
38.  
39.         private void TestReceivers(SPWeb web)
40.         {
41.             int errorCount = 0;
42.             int warningCount = 0;
43.             foreach (SPList list in web.Lists)
44.             {
45.                 Console.WriteLine("INFO: Checking receivers on list: '{0}'", list.Title);
46.                 bool foundReceiver = false;
47.  
48.                 List<String> definitions = new List<String>();
49.  
50.                 foreach (SPEventReceiverDefinition eventReceiver in list.EventReceivers)
51.                 {
52.                     SPEventReceiverType eventReceiverType = eventReceiver.Type;
53.                     foundReceiver = true;
54.                     Console.WriteLine("INFO: Checking receiver Id: '{0}', Name: '{1}'", eventReceiver.Id, eventReceiver.Name);
55.  
56.                     Assembly assembly = null;
57.  
58.                     if (String.IsNullOrEmpty(eventReceiver.Assembly))
59.                     {
60.                         Console.WriteLine("ERROR: The Assembly property is empty or null");
61.                         errorCount++;
62.                         break;
63.                     }
64.                     if (String.IsNullOrEmpty(eventReceiver.Class))
65.                     {
66.                         Console.WriteLine("ERROR: The Class property is empty or null");
67.                         errorCount++;
68.                         break;
69.                     }
70.                     if ((eventReceiverType.ToString().Substring(0, 4) != "Item") && (eventReceiverType != SPEventReceiverType.EmailReceived))
71.                     {
72.                         Console.WriteLine("WARNING: Checking event type: '{0}' is not supported by this tool", eventReceiverType);
73.                         warningCount++;
74.                         break;
75.                     }
76.  
77.                     try
78.                     {
79.                         assembly = Assembly.Load(eventReceiver.Assembly);
80.                     }
81.                     catch (FileNotFoundException ex)
82.                     {
83.                         // general exception data
84.                         Console.WriteLine("ERROR: Loading of assembly '{0}' is failed. An attempt to access a file that does not exist on disk or you have no access to fails.", eventReceiver.Assembly);
85.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
86.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
87.                         // specific exception data
88.                         Console.WriteLine("  File name: '{0}'", ex.FileName);
89.                         Console.WriteLine("  Fusion log: '{0}'", ex.FusionLog);
90.                         errorCount++;
91.                         break;
92.                     }
93.                     catch (FileLoadException ex)
94.                     {
95.                         // general exception data
96.                         Console.WriteLine("ERROR: Loading of assembly '{0}' is failed. The managed assembly is found but cannot be loaded.", eventReceiver.Assembly);
97.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
98.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
99.                         // specific exception data
100.                         Console.WriteLine("  File name: '{0}'", ex.FileName);
101.                         Console.WriteLine("  Fusion log: '{0}'", ex.FusionLog);
102.                         errorCount++;
103.                         break;
104.                     }
105.                     catch (BadImageFormatException ex)
106.                     {
107.                         // general exception data
108.                         Console.WriteLine("ERROR: Loading of assembly '{0}' is failed. The managed assembly is found but cannot be loaded.", eventReceiver.Assembly);
109.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
110.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
111.                         // specific exception data
112.                         Console.WriteLine("  File name: '{0}'", ex.FileName);
113.                         Console.WriteLine("  Fusion log: '{0}'", ex.FusionLog);
114.                         errorCount++;
115.                         break;
116.                     }
117.                     catch (Exception ex)
118.                     {
119.                         // general exception data
120.                         Console.WriteLine("ERROR: Loading of assembly '{0}' is failed", eventReceiver.Assembly);
121.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
122.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
123.                         errorCount++;
124.                         break;
125.                     }
126.  
127.                     Console.WriteLine("INFO: Assembly '{0}' is loaded successfully", eventReceiver.Assembly);
128.  
129.                     Type type = null;
130.                     try
131.                     {
132.                         type = assembly.GetType(eventReceiver.Class);
133.                     }
134.                     catch (FileNotFoundException ex)
135.                     {
136.                         // general exception data
137.                         Console.WriteLine("ERROR: Loading of type {0} from a dependant assembly is failed. An attempt to access a file that does not exist on disk or you have no access to fails.", eventReceiver.Class);
138.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
139.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
140.                         // specific exception data
141.                         Console.WriteLine("  File name: '{0}'", ex.FileName);
142.                         Console.WriteLine("  Fusion log: '{0}'", ex.FusionLog);
143.                         errorCount++;
144.                         break;
145.                     }
146.                     catch (FileLoadException ex)
147.                     {
148.                         // general exception data
149.                         Console.WriteLine("ERROR: Loading of type {0} from a dependant assembly is failed. The managed assembly is found but cannot be loaded.", eventReceiver.Class);
150.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
151.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
152.                         // specific exception data
153.                         Console.WriteLine("  File name: '{0}'", ex.FileName);
154.                         Console.WriteLine("  Fusion log: '{0}'", ex.FusionLog);
155.                         errorCount++;
156.                         break;
157.                     }
158.                     catch (BadImageFormatException ex)
159.                     {
160.                         // general exception data
161.                         Console.WriteLine("ERROR: Loading of type {0} from a dependant assembly is failed. The managed assembly is found but cannot be loaded.", eventReceiver.Class);
162.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
163.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
164.                         // specific exception data
165.                         Console.WriteLine("  File name: '{0}'", ex.FileName);
166.                         Console.WriteLine("  Fusion log: '{0}'", ex.FusionLog);
167.                         errorCount++;
168.                         break;
169.                     }
170.                     catch (Exception ex)
171.                     {
172.                         // general exception data
173.                         Console.WriteLine("ERROR: Loading of type {0} is failed.", eventReceiver.Class);
174.                         Console.WriteLine("  Exception type: '{0}'", ex.GetType());
175.                         Console.WriteLine("  Exception message: '{0}'", ex.Message);
176.                         errorCount++;
177.                         break;
178.                     }
179.  
180.                     if (type == null)
181.                     {
182.                         Console.WriteLine("ERROR: Loading of type {0} is failed.", eventReceiver.Class);
183.                         errorCount++;
184.                         break;
185.                     }
186.  
187.                     Console.WriteLine("INFO: Type '{0}' is loaded successfully", eventReceiver.Class);
188.  
189.                     Type[] eventReceiverMethodParams = null;
190.                     MethodInfo mi = null;
191.                     String methodName = String.Empty;
192.                     bool hadError = false;
193.  
194.                     if (eventReceiverType.ToString().Substring(0, 4) == "Item")
195.                     {
196.                         // checking base class
197.                         if (!type.IsSubclassOf(typeof(SPItemEventReceiver)))
198.                         {
199.                             Console.WriteLine("ERROR: '{0}' class in assembly '{1}' must be subclass of SPItemEventReceiver class'",
200.                                 eventReceiver.Class, eventReceiver.Assembly);
201.                             hadError = true;
202.                             errorCount++;
203.                         }
204.                         else
205.                         {
206.                             methodName = eventReceiverType.ToString();
207.                             eventReceiverMethodParams = new Type[1] { typeof(SPItemEventProperties) };
208.                             mi = type.GetMethod(methodName, BindingFlags.Instance | BindingFlags.Public, null, eventReceiverMethodParams, null);
209.                         }
210.                     }
211.                     else if (eventReceiverType == SPEventReceiverType.EmailReceived)
212.                     {
213.                         // checking base class
214.                         if (!type.IsSubclassOf(typeof(SPEmailEventReceiver)))
215.                         {
216.                             Console.WriteLine("ERROR: '{0}' class in assembly '{1}' must be subclass of SPEmailEventReceiver class'",
217.                                 eventReceiver.Class, eventReceiver.Assembly);
218.                             hadError = true;
219.                             errorCount++;
220.                         }
221.                         else
222.                         {
223.                             methodName = "EmailReceived";
224.                             eventReceiverMethodParams = new Type[3] { typeof(SPList), typeof(SPEmailMessage), typeof(String) };
225.                             mi = type.GetMethod(methodName, BindingFlags.Instance | BindingFlags.Public, null, eventReceiverMethodParams, null);
226.                         }
227.                     }
228.                     if (!hadError)
229.                     {
230.                         if (mi == null)
231.                         {
232.                             Console.WriteLine("ERROR: '{0}' class in assembly '{1}' does not implement the method '{2}' required to handle event type '{3}'",
233.                                 eventReceiver.Class, eventReceiver.Assembly, methodName, eventReceiverType);
234.                             hadError = true;
235.                             errorCount++;
236.                         }
237.                         else if (mi.DeclaringType != type)
238.                         {
239.                             Console.WriteLine("WARNING: Only a base class of the '{0}' class in assembly '{1}' does implement the method '{2}' to handle event type '{3}'. Probably receiver class does not override the method declaration of the base class.",
240.                                 eventReceiver.Class, eventReceiver.Assembly, methodName, eventReceiverType);
241.                             hadError = true;
242.                             warningCount++;
243.  
244.                         }
245.                         else
246.                         {
247.                             String definition = String.Format("{0};#{1}", type.FullName, eventReceiverType);
248.                             if (definitions.Contains(definition))
249.                             {
250.                                 Console.WriteLine("WARNING: '{0}' class in assembly '{1}' is registered multiple times to the method '{2}' to handle event type '{3}' in the same list",
251.                                     eventReceiver.Class, eventReceiver.Assembly, methodName, eventReceiverType);
252.                                 warningCount++;
253.                             }
254.                             else
255.                             {
256.                                 Console.WriteLine("INFO: Checking '{0}' event type is successful", eventReceiverType);
257.                                 definitions.Add(definition);
258.                             }
259.                         }
260.                     }
261.                 }
262.  
263.                 if (!foundReceiver)
264.                 {
265.                     Console.WriteLine("INFO: No receiver found in this list");
266.                 }
267.             }
268.             Console.WriteLine("Error(s): {0}, Warning(s): {1}", errorCount, warningCount);
269.         }
270.     }
271. }

Although the example above works only with the most common event types, it provides you a sample how to do more advanced checks for assemblies, classes and methods.

How to change the order of the fields on a SharePoint form using code

When you add a new field to a list, it is automatically placed at the last position of the form. It is not always what you want.

On the user interface you can use the Column ordering link from the list settings page, but there is no direct support in the object model to set the position of a field.

You can find a few examples on the web about using the REORDERFIELDS method:

• Reorder SharePoint list fields from code
• The gem of SPWeb – ProcessBatchData
• Export, Import, and Update List Fields

I found these example a bit overcomplicated and inconsistent, lot of unnecessary tricks with web and list references and IDs, building part of the XML request using XmlTextWriter, other part as string, etc. It is not surprise that lot of comments complain about different type of exceptions when testing the code.

The above posts do not bother too much with the theory of reordering although it would help better understanding of code and finding the source of the error easier.

The SPField class has a property called Reorderable. Reorderable fields are displayed on the Change Field Order page. These fields are available only for reordering but you should include all fields of the list in the request. Reorderable fields come first and non-reorderable fields must follow them. You should refer to the fields using their internal name. Double check the names to be sure you use the correct ones. Naming is not always trivial, for example, the internal name of the Description field in the Tasks list is Body.

Michael Ekegren wrote: ‘The version is very important to remember otherwise you’ll get an error message like” the list have been updated by another use – click back in the browser to refresh your changes” or something like that.’ Well, that is not exact. Based on the documentation and my experience, the owshiddenversion attribute is optional. Of course, if you use that, you must set its value correctly. But if you omit that, you also get an error message, if the list is changed in the meantime.

The following code sample shows how to move a field to the last position.

1. String fieldReorderable = String.Empty;
2. String fieldNonReorderable = String.Empty;
3. String fieldStrFormat = "<Field Name=\"{0}\" />\r\n";
4. // we move this field to the last position
5. String fieldNameToLast = "Body";
6.  
7. string reorderMethodFormat = @"<?xml version=""1.0"" encoding=""UTF-8""?>  
8.         <Method ID=""0"">  
9.         <SetList Scope=""Request"">{0}</SetList>  
10.         <SetVar Name=""Cmd"">REORDERFIELDS</SetVar>  
11.         <SetVar Name=""ReorderedFields"">{1}</SetVar>   
12.         </Method>";
13.  
14. foreach (SPField field in list.Fields)
15. {
16.     if ((field.Reorderable) && (field.InternalName != fieldNameToLast))
17.     {
18.         fieldReorderable += String.Format(fieldStrFormat, field.InternalName);
19.     }
20.     else if (!field.Reorderable)
21.     {
22.         fieldNonReorderable += String.Format(fieldStrFormat, field.InternalName);
23.     }
24. }
25.  
26. String fieldMoved = String.Format(fieldStrFormat, fieldNameToLast);
27. String fields = String.Format("<Fields>{0}{1}{2}</Fields>", fieldReorderable, fieldMoved, fieldNonReorderable);
28.  
29. String reorder = String.Format(reorderMethodFormat, list.ID, SPHttpUtility.HtmlEncode(fields));
30. web.AllowUnsafeUpdates = true;
31. String res = web.ProcessBatchData(reorder);
32. web.AllowUnsafeUpdates = false;

The Code attribute of the Result tag in the response message for a REORDERFIELDS request must be zero on success or a negative value on error.

Some typical error you can face when working with this code:

<Result ID="0" Code="-2147467259">
<ErrorText>Cannot complete this action.

Please try again.</ErrorText></Result>

The format of the request is wrong. Be sure you encoded the value set for the ReorderedFields variable.

<Result ID="0" Code="-2130575323">
<ErrorText>Fields have been added or removed since you began this editing session. Please refresh your view and try again.</ErrorText></Result>

There are missing or extra fields in the request. Be sure you used the proper internal names.

<Result ID="0" Code="-2130575305">
<ErrorText>Save Conflict

Your changes conflict with those made concurrently by another user. If you want your changes to be applied, click Back in your Web browser, refresh the page, and resubmit your changes.</ErrorText></Result>

The schema of the target list has been changed in the meantime.

Finally here are a few exceptions and the possible reasons.

SPException when calling ProcessBatchData: “The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again." You probably forgot to set the AllowUnsafeUpdates property of the current SPWeb instance to true.

"Exception has been thrown by the target of an invocation.", InnerException: "Object reference not set to an instance of an object." in the SPField variable declaration part of the foreach block. The field is not installed correctly. Try to remove and add again. If you add the field from code, check the code.

ArgumentException when calling ProcessBatchData: "Value does not fall within the expected range." You should use double quotes, not apostrophes in the XML header.

Correct:

<?xml version="1.0" encoding="UTF-8"?>

Wrong:

<?xml version=’1.0′ encoding=’UTF-8′?>

Updating multi value fields using web service call and batch update

In my recent post I showed some examples about updating lookup (SPFieldLookup) and user (SPFieldUser) fields. The examples in that post worked with single value fields.

Updating multi value fields through web service calls or using batch updates is a bit different and a rather undocumented area of SharePoint development, so let’s see some examples about that.

I’ve altered the Tasks list we worked with in the past article to allow its Location field to have multiple value, furthermore I’ve added a multi value user field called Owners, and a multi value choice field called Choices with options Value1, Value2 and Value3. It means the value of the Location field will be of type SPFieldLookupValueCollection, the value of the Owners field will be of type SPFieldUserValueCollection, and the value of the Choices field will be of type SPFieldMultiChoiceValue.

The following code example shows how to update these fields using a batch that updates the list item having ID = 3 by calling the ProcessBatchData method.

1. SPList list = web.Lists["Tasks"];
2.  
3. String batch = String.Format("<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
4.     "<ows:Batch OnError=\"Continue\"><Method ID='1'>" +
5.     "<SetList>{0}</SetList>" +
6.     "<SetVar Name='Cmd'>Save</SetVar>" +
7.     "<SetVar Name='ID'>3</SetVar>" +
8.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Location'>1;#;#2</SetVar>" +
9.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Owners'>1;#;#7<</SetVar>" +
10.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Choices'>Value1;#Value2</SetVar>" +
11.     "</Method></ows:Batch>", list.ID);
12.  
13. String batchResult = list.ParentWeb.ProcessBatchData(batch);

The main point is that you should separate IDs in lookup and user fields with “;#;#”, but you should use only a simple “;#” in the case of the multi choice field. That is because we can omit the lookup value of the field value, and you should only specify the lookup ID. As we discussed in the former post, you can specify an arbitrary lookup value if you wish, like “1;#some value;#2;#other value” or “1;#user1;#7;#user7” it has no effect to the update.

An important note about an issue I’ve experienced when playing with the code. As you may have already noticed I prefer using apostrophes to double quotation marks as they make it possible to create more readable code. If you replaces the quotes in the XML header tag as well as shown below:

<?xml version=’1.0′ encoding=’UTF-8′?>

You will get an exception that states:

Value does not fall within the expected range.

At first one can think it is because of a misspelled field name but the real reason is quite different. So keep using quotes in the header.

The following code illustrates the very same concept using UpdateListItems method of the Lists web service, the only difference is that in this case we update the list item having ID = 2.

1. MyListService.Credentials = CredentialCache.DefaultCredentials;
2. MyListService.Url = "http://yourserver/_vti_bin/lists.asmx&quot;;
3.  
4. XmlDocument updateRequest = new XmlDocument();
5.  
6. String updateBatch = "<Batch OnError='Continue'>" +
7.                         "<Method ID='1' Cmd='Update'>" +
8.                         "<Field Name='ID'>2</Field>" +
9.                         "<Field Name='Location'>1;#;#2</Field>" +
10.                         "<Field Name='Owners'>1;#;#7</Field>" +
11.                         "<Field Name='Choices'>Value1;#Value2</Field>" +
12.                         "</Method>" +
13.                         "</Batch>";
14.  
15. updateRequest.LoadXml(updateBatch);
16.  
17. XmlNode deleteResult = MyListService.UpdateListItems("Tasks", updateRequest.DocumentElement);

Updating SharePoint lookup and user field values

When working with SharePoint list you should often update field values, like lookup fields (SPFieldLookup) and user fields (SPFieldUser, that is a subclass of the former one). In these cases you have to work with value types of SPFieldLookupValue (or SPFieldLookupValueCollection for multi valued fields) or SPFieldUserValue (SPFieldUserValueCollection for multi valued fields).

If you want to update these type of fields, you should know the ID of the referenced list item or user only. Jinal Patel shows an example for that in this post using the UpdateListItems method of the Lists web service.

You can do the same when updating the items using the object model either through updating an individual item or using the batch update method.

For the sake of this example I’ve added a single value lookup field called Location to a Task list on my site. This field refers to another list of the same site that contains city names in the referenced field.

The following code updates the Location field (SPFieldLookup) and the AssignedTo field (SPFieldUser) of the task list item having ID = 1.

1. SPList list = web.Lists["Tasks"];
2. SPListItem item = list.Items.GetItemById(1);
3. item["Location"] = new SPFieldLookupValue(1, null);
4. item["AssignedTo"] = new SPFieldUserValue(web, 1, null);
5. item.Update();

Including the name part of the lookup field has really no effect; it is simply ignored, for example:

item["Location"] = new SPFieldLookupValue(1, "abcdefgh");

The above code has the same result as using null as the lookup name, even if the name part is not matching the real value of the remote field.

You can do the same using the ProcessBatchData method. The following example updates the task list item having ID = 3.

1. String batch = String.Format("<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
2.     "<ows:Batch OnError=\"Continue\"><Method ID='1'>" +
3.     "<SetList>{0}</SetList>" +
4.     "<SetVar Name='Cmd'>Save</SetVar>" +
5.     "<SetVar Name='ID'>3</SetVar>" +
6.     "<SetVar Name='urn:schemas-microsoft-com:office:office#Location'>2</SetVar>" +
7.     "<SetVar Name='urn:schemas-microsoft-com:office:office#AssignedTo'>1</SetVar>" +
8.     "</Method></ows:Batch>", list.ID);
9. String batchResult = list.ParentWeb.ProcessBatchData(batch);

Including the name part of the lookup field has no effect as in the case of the single item update above.

<SetVar Name=’urn:schemas-microsoft-com:office:office#Location’>2;#abcdefgh</SetVar>

This would be identical with the previous version of batch update without the name part, even if the name part is not matching the real value of the remote field.

This behavior of the lookup and user fields is very nice, as if it required the name part as well would mean we should do an extra lookup in the remote list for the value of the referenced field.