Second Life of a Hungarian SharePoint Geek

April 24, 2013

Creating a mail distributor system using the incoming mail feature of SharePoint

Filed under: Incoming email, Reflection, SP 2010 — Tags: , , — Peter Holpar @ 21:56

Wouldn’t it be great to implement your own custom logic to distribute mails to targeted addresses (for example, based on sender or subject of the mail) using SharePoint lists and event receivers? In this post I show you the fundamental technical issues and their solution to achieve that goal. My “custom logic” is quite simple, I send the same mail back to the sender, however you can build more sophisticated logic using the same technique, but more on that later.

Before starting Visual Studio, I’ve created a simple custom list called MailDistributor on my SharePoint site.

In Visual Studio I chose the Empty SharePoint Project template, and added a new List Email Event event receiver item.

Having these artifacts, I altered the default Elements.xml, to register the event receiver to the list I created earlier:

Code Snippet
  1. <?xml version="1.0" encoding="utf-8"?>
  2. <Elements xmlns="">
  3.     <Receivers ListUrl="Lists/MailDistributor">
  4.         <Receiver>
  5.             <Name>IncomingMailHandlerEmailReceived</Name>
  6.             <Type>EmailReceived</Type>
  7.             <Assembly>$SharePoint.Project.AssemblyFullName$</Assembly>
  8.             <Class>MailDistributor.IncomingMailHandler</Class>
  9.             <SequenceNumber>10000</SequenceNumber>
  10.         </Receiver>
  11.     </Receivers>
  12. </Elements>

Regarding the code, the first step was to create the extension method GetMailMessage for the SPEmailMessage type to convert it to a MailMessage object (System.Net.Mail namespace). Fortunately, both of these object types have the same stream format in the background. This stream is directly accessible from the SPEmailMessage, however, MailMessage  is not creatable from the stream (or ByteArray / String) format. To solve this issue, I utilized the RxMailMessage type (copyright by Peter Huber, Singapore), that is a derived class of MailMessage with Stream and File support.

Code Snippet
  1. public static MailMessage GetMailMessage(this SPEmailMessage spEmailMessage)
  2. {
  3.     MailMessage result = null;
  4.     if (spEmailMessage != null)
  5.     {
  6.         result = RxMailMessage.CreateFromStream(spEmailMessage.GetMessageStream());
  7.     }
  9.     return result;
  10. }

Below is the structure of the solution, highlighted the classes borrowed from Peter Huber.


The next challenge was to set the addressee (To property) of the MailMessage instance. Since there is no way to change this read-only property using the public methods of the type (in practice, you should set it already in the constructor), I had to apply my experience with Reflection, and set the private field to of the private field message of the MailMessage instance. Although I set only the To property in the example using the SetTo extension method, you could (and should!) set the Cc and Bcc fields as well. For example, clear these values to avoid perpetual sending / receiving the same message in the case the one of the Cc / Bcc fields contain the incoming mail address of the SharePoint list. To do that, you should implement the SetCc and SetBcc methods and from these methods call the SetMailAddressCollection method with the parameters “cc” and “bcc” accordingly.

Code Snippet
  1. public static void SetTo(this MailMessage mailMessage, MailAddressCollection mailAddressCollection)
  2. {
  3.     if ((mailMessage != null) && (mailAddressCollection != null))
  4.     {
  5.         SetMailAddressCollection(mailMessage, mailAddressCollection, "to");
  6.     }
  7. }
  9. private static void SetMailAddressCollection(MailMessage mailMessage, MailAddressCollection mailAddressCollection, string fieldName)
  10. {
  11.     if ((mailMessage != null) && (mailAddressCollection != null) && (fieldName != null))
  12.     {
  13.         Type typeMailMessage = typeof(MailMessage);
  15.         FieldInfo fi = typeMailMessage.GetField("message", BindingFlags.NonPublic | BindingFlags.Instance);
  17.         if (fi != null)
  18.         {
  19.             object message = fi.GetValue(mailMessage);
  21.             if (message != null)
  22.             {
  23.                 Type typeMessage = message.GetType(); // it is internal class System.Net.Mail.Message
  25.                 FieldInfo fi2 = typeMessage.GetField(fieldName, BindingFlags.NonPublic | BindingFlags.Instance);
  27.                 if (fi2 != null)
  28.                 {
  29.                     fi2.SetValue(message, mailAddressCollection);
  30.                 }
  31.             }
  32.         }
  33.     }
  34. }

In the event receiver, we convert the SPEmailMessage into a MailMessage instance, set its To property to the e-mail address of the original poster (From property of the MailMessage) and send the mail using an SmtpClient object. The Host property of the SmtpClient instance can be set using the Address of the configured SMTP server of the OutboundMailServiceInstance in the current web application.

As you can see, most of the code below is just tracing to help us to follow the process using DebugView. You are free to remove these lines without affecting the functionality, of course.

Code Snippet
  1. using System;
  2. using System.Diagnostics;
  3. using System.Net.Mail;
  4. using Microsoft.SharePoint;
  5. using Microsoft.SharePoint.Utilities;
  7. namespace MailDistributor
  8. {
  9.     public class IncomingMailHandler : SPEmailEventReceiver
  10.     {
  11.         public override void EmailReceived(SPList list, SPEmailMessage emailMessage, string receiverData)
  12.         {
  13.             try
  14.             {
  15.                 Trace.TraceInformation("IncomingMailHandler starting…");
  17.                 foreach (SPEmailHeader header in emailMessage.Headers)
  18.                 {
  19.                     Trace.TraceInformation("EmailReceived emailMessage header {0}, {1}", header.Name, header.Value);
  20.                 }
  22.                 SmtpClient smtpClient = new SmtpClient();
  24.                 smtpClient.Host = list.ParentWeb.Site.WebApplication.OutboundMailServiceInstance.Server.Address;
  26.                 Trace.TraceInformation("IncomingMailHandler: getting mail message from stream");
  28.                 MailMessage mailMessage = emailMessage.GetMailMessage();
  30.                 Trace.TraceInformation("IncomingMailHandler: setting mail message To field");
  32.                 mailMessage.SetTo(new MailAddressCollection { mailMessage.From });
  34.                 Trace.TraceInformation("IncomingMailHandler: sending mail");
  36.                 smtpClient.Send(mailMessage);
  38.                 Trace.TraceInformation("IncomingMailHandler: finished");
  39.             }
  40.             catch (Exception ex)
  41.             {
  42.                 Trace.TraceInformation("IncomingMailHandler exception: {0}", ex.Message);
  43.             }
  45.         }
  47.     }
  49. }

After deploying the solution and activating the feature, we should enable the incoming mail for the MailDistributor list, set the mail address alias, and send a test message to this address. If there is no error, within about a minute we should receive the same mail, including formatting and attachments to the mailbox of the sender.

Using the Category settings of the mail to route the message

The rule we implemented is really a trivial one and has not much sense, but one can implement more complicated and more useful routing rules as well. My plan is to build a routing “engine” based on the Category settings of the mail.

As part of the Options / Tracking properties in Outlook, we can set not only Blue or Green categories, but our own custom categories (like SharePoint and Silverlight below) as well. As long as these categories are transferred within the mail, we can process them in our event receiver, look up SharePoint user profiles having the same values set in the Ask me about property, and route the message exactly to those users, implementing thus a simple but efficient knowledge management solution.

It would be even better if the user could choose those category values from a SharePoint Managed Metadata keyword list (a candidate for an Office 2013 mail-app?).


However there are some issues with the Category property that you should be aware of.

Based on this information, Outlook removes category settings from outgoing mails due to privacy concerns. One can alter this settings via registry (HKEY_CURRENT_USER\Software\Policies\Microsoft\Office\xx.0\Outlook\Preferences\SendPersonalCategories, where xx is the version number of Outlook, like 14 for Outlook 2010).

Exchange 2010 also removes the categories from the outgoing messages by default, as I’ve learned here. This behavior can be changes using the following PowerShell command:

Set-TransportConfig -ClearCategories:$False

A workaround for these issues might be an Outlook add-in, or a simply VBA code like this one, that illustrates, how to copy the mail categories to a custom mail header called X-Categories when sending the mail, thus avoiding losing of the categories:

Private Sub Application_ItemSend(ByVal item As Object, Cancel As Boolean)
    Dim mi As MailItem
    Set mi = item
    If Not mi Is Nothing Then
       item.PropertyAccessor.SetProperty "{00020386-0000-0000-C000-000000000046}/X-Categories", item.Categories
    End If
End Sub

Of course, we should filter the mails affected by this behavior based on the To e-mail address, limiting it to the mails sent to our MailDistributor list.

October 26, 2012

How to populate the Attendees field of a SharePoint event based on the addressees of a meeting request? (Version 2)

In my previous post I already demonstrated a method to resolve the meeting attendees based on the mail addresses in the incoming mail, though – as I wrote there – that method has issues with event updates.

Note: In this post I show you an alternative, that – at least, based on my experience – performs better. However, the code below uses non-public API calls and accesses SharePoint database directly, so it is not a supported approach. Use this sample at you own risk and preferably only in test environments.

In this version of implementation we alter the standard pipeline of incoming mail processing for our calendar to inject our code into the process. To achieve that, we create an a SPEmailHandler that first invokes the ProcessMessage method of the SPCalendarEmailHandler class to achieve the standard functionality, then resolves the attendees using the To mail header property based on this technique, and updates the related item / all related items (in the case of a recurring event, it may be not a single item) in the list.

Note: In the case of the To mail header property we don’t need to unescape the value, so you should comment out this line of code in the GetUsersByMailTo method introduced in the first part of this post:

emailTo = emailTo.Replace("&lt;", "<").Replace("&gt;", ">");

We can get the corresponding item(s) based the unique identifier (UID, you can read details on Wikipedia) vCalendar property of the event using the GetCalendarProp method described in my former post. We use the GetExistingItems method below to get these related items:

  1. private SPListItemCollection GetExistingItems(SPList list, string uid)
  2. {            
  3.     SPQuery query = new SPQuery();
  4.     query.Query = "<Where><Eq><FieldRef Name=\"" + list.Fields[SPBuiltInFieldId.EmailCalendarUid].InternalName + "\"/><Value Type=\"Text\">" + SPEncode.HtmlEncode(uid) + "</Value></Eq></Where>";
  5.     SPListItemCollection existingItems = list.GetItems(query);
  6.     return existingItems;
  7. }

The following method allows us to invoke the ProcessMessage method of the SPCalendarEmailHandler class:

  1. private void ProcessMessage(SPList list, SPEmailMessage emailMessage)
  2. {
  3.     Trace.TraceInformation("Starting SPCalendarEmailHandler processing");
  5.     string spCalendarEmailHandlerTypeName = "Microsoft.SharePoint.SPCalendarEmailHandler";
  7.     // hack to get the Microsoft.SharPoint assembly
  8.     Assembly sharePointAssembly = typeof(SPWeb).Assembly;
  9.     // and a reference to the type of the SPCalendarEmailHandler internal class
  10.     Type spCalendarEmailHandlerType = sharePointAssembly.GetType(spCalendarEmailHandlerTypeName);
  12.     // spCalendarEmailHandler will be of type internal class
  13.     // Microsoft.SharePoint.SPCalendarEmailHandler
  14.     // defined in Microsoft.SharePoint assembly
  15.     object spCalendarEmailHandler = sharePointAssembly.CreateInstance(spCalendarEmailHandlerTypeName, false,
  16.         BindingFlags.Public | BindingFlags.Instance, null, new object[] { list }, CultureInfo.InvariantCulture, null);
  18.     if (spCalendarEmailHandler != null)
  19.     {
  20.         MethodInfo mi_ProcessMessage = spCalendarEmailHandlerType.GetMethod("ProcessMessage",
  21.                     BindingFlags.Public | BindingFlags.Instance, null,
  22.                     new Type[] { typeof(SPEmailMessage) }, null
  23.                     );
  24.         if (mi_ProcessMessage != null)
  25.         {
  26.             // result of type SPEmailHandlerResult is ignored
  27.             mi_ProcessMessage.Invoke(spCalendarEmailHandler, new Object[] { emailMessage });
  28.         }
  29.     }
  31.     Trace.TraceInformation("SPCalendarEmailHandler processing finished");
  32. }

Using these helper methods our EmailReceived method looks like these:

  1. public override void EmailReceived(SPList list, SPEmailMessage emailMessage, string receiverData)
  2. {
  3.     try
  4.     {
  5.         Trace.TraceInformation("EmailReceived started");
  7.         string uid = GetCalendarProp(emailMessage, "UID");
  9.         ProcessMessage(list, emailMessage);
  11.         string emailTo = emailMessage.Headers["To"];
  12.         SPFieldUserValueCollection users = GetUsersByMailTo(list.ParentWeb, emailTo);
  14.         if (!string.IsNullOrEmpty(uid))
  15.         {
  16.             SPListItemCollection existingItems = GetExistingItems(list, uid);
  17.             foreach (SPListItem listItem in existingItems)
  18.             {
  19.                 Trace.TraceInformation("Updating item ID: {0}, To: {1}", listItem.ID, emailTo);
  20.                 listItem[SPBuiltInFieldId.ParticipantsPicker] = users;
  21.                 listItem.Update();
  22.             }
  23.         }
  25.         Trace.TraceInformation("EmailReceived calling base handler(s)…");
  26.     }
  27.     catch (Exception ex)
  28.     {
  29.         Trace.TraceInformation("EmailReceived exception: {0}", ex.Message);
  30.         Trace.TraceInformation(ex.StackTrace);
  31.     }
  32.     base.EmailReceived(list, emailMessage, receiverData);
  33. }

Finally, this method seems to fulfill our goals and resolves attendees both on new meeting requests and event updates.

How to populate the Attendees field of a SharePoint event based on the addressees of a meeting request? (Version 1)

Filed under: Calendar, Event receivers, Incoming email, SP 2010 — Tags: , , , — Peter Holpar @ 09:39

As I formerly wrote, if you enable the incoming mails on a SharePoint calendar, and send a meeting request to the list, the participants’ mail addresses won’t be resolved to meeting attendees.

To workaround this limitation, my first idea was a SPEmailHandler that extracts this info from the mail and stores it into the adequate SharePoint field. However I found, that after I registered my event receiver on the list, all of the default functionality of the incoming mail on calendars (like resolving time and location of the meetings, updating former items in the list on event updates, etc.) were lost, even if I allow in my override of EmailReceived method other registered receivers to be called, like:

base.EmailReceived(list, emailMessage, receiverData);

The reason for this phenomena I have found reflecting the related classes and in this post. That means, specific list types, like calendars, announcements, discussions, etc. have their standard ProcessMessage methods implemented in subclasses of the internal abstract SPEmailHandler class. For example, the incoming mails for a calendar is handled by the SPCalendarEmailHandler class. If you register a custom event receiver (that means the HasExternalEmailHandler property of the list will be true), the standard method will be totally ignored, and the custom event receiver will be called through a SPExternalEMailHandler instance instead. Really bad news!

How could we inject our requirement of resolving attendees into this processing chain without disturbing the standard steps?

Spending a few minutes with Reflector I found, that the SetStandardHeaderFields method of the SPEmailHandler class (called from the ProcessVEvent method of the SPCalendarEmailHandler class) sets (among others) the EmailTo field, and the SetEmailHeadersField method of the same class (called from the SetStandardHeaderFields method) sets the EmailHeaders field.

Both of these fields seem to be undocumented hidden fields. The EmailHeaders field contains the whole (unescaped!) SMTP header block of the original mail that triggered the item creation, while the EmailTo field contains the addresses of the mail in these escaped format:

John Smith &lt;;;  Peter Black &lt;;

The escaping seems to be important. When I tried setting the values without it, the e-mail addresses were removed from the field, leaving only the display names there.

To resolve users, I used a slightly modified version of the method demonstrated in my former post:

  1. private SPFieldUserValueCollection GetUsersByMailTo(SPWeb web, string emailTo)
  2. {
  3.     SPFieldUserValueCollection result = new SPFieldUserValueCollection();
  4.     if (!string.IsNullOrEmpty(emailTo))
  5.     {
  6.         emailTo = emailTo.Replace("&lt;", "<").Replace("&gt;", ">");
  7.         string[] addressees = emailTo.Split(';');
  9.         Array.ForEach(addressees,
  10.             addressee =>
  11.             {
  12.                 MailAddress ma = new MailAddress(addressee);
  13.                 SPPrincipalInfo pi = SPUtility.ResolveWindowsPrincipal(web.Site.WebApplication, ma.Address, SPPrincipalType.User, true);
  14.                 if ((pi != null) && (!string.IsNullOrEmpty(pi.LoginName)))
  15.                 {
  16.                     SPUser user = web.EnsureUser(pi.LoginName);
  17.                     result.Add(new SPFieldUserValue(web, user.ID, null));
  18.                     Console.WriteLine("User: {0}", user.LoginName);
  19.                 }
  20.             });
  21.     }
  23.     return result;
  24. }

My next step was to create an ItemUpdated event handler that should resolve the users based on their e-mail addresses. An issue I had to handle there is, that the item can be updated not only due to the incoming mail, but also due to the changes the users make through the UI. It would be rather frustrating for our users, if we set the original value back, after the they altered it through the UI. So I triggered the user resolving process only if the value of the EmailTo was not empty, and clear the value of the field after processing to prohibit further calls of the method (for example, triggered from the UI).

Note: To update the attendees of the event, we should set the value of the ParticipantsPicker field, and not the Participants field.

  1. public override void ItemUpdated(SPItemEventProperties properties)
  2. {
  3.     try
  4.     {
  5.         Trace.TraceInformation("ItemUpdated started");
  6.         SPListItem listItem = properties.ListItem;
  7.         string emailTo = listItem[SPBuiltInFieldId.EmailTo] as string;
  8.         Trace.TraceInformation("emailTo: {0}", emailTo);
  9.         SPWeb web = properties.Web;
  11.         if (!string.IsNullOrEmpty(emailTo))
  12.         {
  13.             Trace.TraceInformation("Updating attendees");
  14.             listItem[SPBuiltInFieldId.ParticipantsPicker] = GetUsersByMailTo(web, emailTo);
  15.             // hack to prohibit triggering on updates from UI
  16.             listItem[Microsoft.SharePoint.SPBuiltInFieldId.EmailTo] = null;
  17.             listItem.Update();
  18.         }
  19.         Trace.TraceInformation("ItemUpdated finished");
  20.     }
  21.     catch (Exception ex)
  22.     {
  23.         Trace.TraceInformation("ItemUpdated exception: {0}", ex.Message);
  24.         Trace.TraceInformation(ex.StackTrace);
  25.     }
  26.     base.ItemUpdated(properties);
  27. }

After deploying my event receiver I’ve checked it through sending meeting requests to attendees including the address of the calendar list, and first the solution seemed to be perfect. However, after a while I found, that when I send an update for an event, the list of the attendees was not updated.

The reason behind this issue is – as it turned out after another round of reflectioning – that the EmailTo field is set only once, when the item is created, but not populated when updates are received. See the output parameter isUpdate of the FindItemToUpdate method of the SPCalendarEmailHandler class for details. If the actual processing is an update, the SetStandardHeaderFields method, and through this method the SetEmailHeadersField method won’t be invoked in the ProcessVEvent method.

In the next post I try to publish the alternative solution for the original request – resolving meeting attendees based on the mail addresses in the incoming mail.

October 15, 2012

How to list all SharePoint incoming mail aliases for a site or web application?

Filed under: Incoming email, Reflection, SP 2010 — Tags: , , — Peter Holpar @ 21:21

In my recent samples I’ve illustrated how to check whether an alias is reserved and how to get details of mapping.

This time I provide you the code that helps to enumerate all aliases for a site or the entire web application. To use this code you need the EmailAliasRecord wrapper struct and the extension methods as introduced in the former post.

Note: the code below uses non-public API calls and accesses SharePoint database directly, so it is not a supported approach. Use this sample at you own risk and preferably only in test environments.

The information we need is accessible through the proc_EnumEmailAliases and proc_EnumEmailAliasesBySite stored procedure in the SharePoint content database of the give web application.

To access these procedures, I first introduced the ExecuteReader extension method:

  1. public static SqlDataReader ExecuteReader(this SPContentDatabase database, SqlCommand command, CommandBehavior behavior)
  2. {
  3.     SqlDataReader result = null;
  5.     string sqlSessionTypeName = "Microsoft.SharePoint.Utilities.SqlSession";
  6.     Type spContentDatabaseType = typeof(SPContentDatabase);
  8.     // hack to get the Microsoft.SharPoint assembly
  9.     Assembly sharePointAssembly = typeof(SPWeb).Assembly;
  10.     // and a reference to the type of the SqlSession internal class
  11.     Type sqlSessionType = sharePointAssembly.GetType(sqlSessionTypeName);
  13.     System.Reflection.PropertyInfo pi_SqlSession = spContentDatabaseType.GetProperty("SqlSession", BindingFlags.NonPublic | BindingFlags.Instance);
  15.     if (pi_SqlSession != null)
  16.     {
  17.         // sqlSession will be of type internal class
  18.         // Microsoft.SharePoint.Utilities.SqlSession
  19.         // defined in Microsoft.SharePoint assembly
  20.         object sqlSession = pi_SqlSession.GetValue(database, null);
  22.         MethodInfo mi_ExecuteReader = sqlSessionType.GetMethod("ExecuteReader", BindingFlags.Public | BindingFlags.Instance, null,
  23.                                             new Type[] { typeof(SqlCommand), typeof(CommandBehavior) }, null);
  25.         if (mi_ExecuteReader != null)
  26.         {
  27.             result = mi_ExecuteReader.Invoke(sqlSession, new Object[] { command, behavior }) as SqlDataReader;
  28.         }
  29.     }
  31.     return result;
  32. }

Having this method and the code from the previous post, the methods below can be used to display aliases mapped for lists in a specific site, in a content database or in a web application:

  1. private void DisplayEmailAliases(SPWebApplication webApp)
  2. {
  3.     foreach (SPContentDatabase database in webApp.ContentDatabases)
  4.     {
  5.         DisplayEmailAliases(database, null);
  6.     }
  7. }
  9. private void DisplayEmailAliases(SPSite site)
  10. {
  11.     SPContentDatabase database = site.ContentDatabase;
  12.     DisplayEmailAliases(database, site.ID);
  13. }
  15. private void DisplayEmailAliases(SPContentDatabase database, Guid? siteId)
  16. {
  17.     SqlCommand command;
  18.     List<EmailAliasRecord> list = new List<EmailAliasRecord>();
  19.     if (!siteId.HasValue)
  20.     {
  21.         command = new SqlCommand("proc_enumEmailAliases");
  22.     }
  23.     else
  24.     {
  25.         command = new SqlCommand("proc_enumEmailAliasesBySite");
  26.         command.Parameters.Add("@SiteId", SqlDbType.UniqueIdentifier).Value = siteId.Value;
  27.     }
  28.     command.CommandType = CommandType.StoredProcedure;
  30.     using (SqlDataReader reader = database.ExecuteReader(command, CommandBehavior.CloseConnection))
  31.     {
  32.         while (reader.Read())
  33.         {
  34.             EmailAliasRecord ear = new EmailAliasRecord(reader);
  35.             Console.WriteLine(ear.ToString());
  36.         }
  37.     }
  39. }

How to resolve SharePoint incoming mail alias mapping details?

Filed under: Incoming email, Reflection, SP 2010 — Tags: , , — Peter Holpar @ 20:36

In my recent post I’ve illustrated how to check from code whether a given mail alias is already reserved. If you need to know details about the list / web the alias is mapped to, you have to work further on the issue.

Note: the code below uses non-public API calls and so it is not a supported approach. Use this sample at you own risk and preferably only in test environments.

First I introduced a few extension methods to make our live (and work with Reflection) a bit easier.

  1. public static class Extensions
  2. {
  3.     public static object GetPublicInstanceFieldValue(this Type type, string fieldName, object instance)
  4.     {
  5.         object result = null;
  6.         FieldInfo fi = type.GetField(fieldName);
  8.         if ((fi != null) && (instance != null))
  9.         {
  10.             result = fi.GetValue(instance);
  11.         }
  13.         return result;
  14.     }
  16.     public static void SetPublicInstanceFieldValue(this Type type, string fieldName, object fieldValue, object instance)
  17.     {
  18.         FieldInfo fi = type.GetField(fieldName);
  20.         if ((fi != null) && (instance != null))
  21.         {
  22.             fi.SetValue(instance, fieldValue);
  23.         }
  24.     }
  26.     public static object GetPublicInstancePropertyValue(this Type type, string fieldName, object instance)
  27.     {
  28.         object result = null;
  29.         System.Reflection.PropertyInfo pi = type.GetProperty(fieldName);
  31.         if ((pi != null) && (instance != null))
  32.         {
  33.             result = pi.GetValue(instance, null);
  34.         }
  36.         return result;
  37.     }
  38. }

Next, I defined the following wrapper struct for the internal EmailAliasRecord struct (Microsoft.SharePoint.Administration namespace, Microsoft.SharePoint assembly):

  1. public struct EmailAliasRecord
  2. {
  3.     public string Alias { get; private set; }
  4.     public Guid ListId { get; private set; }
  5.     public Guid WebId { get; private set; }
  6.     public Guid SiteId { get; private set; }
  7.     public bool IsValid { get; private set; }
  9.     private const string _emailAliasRecordTypeName = "Microsoft.SharePoint.Administration.EmailAliasRecord";
  11.     public EmailAliasRecord(object emailAliasRecord) : this()
  12.     {
  13.         InitFields(emailAliasRecord);
  14.     }
  16.     private void InitFields(object emailAliasRecord)
  17.     {
  18.         // hack to get the Microsoft.SharPoint assembly
  19.         Assembly sharePointAssembly = typeof(SPWeb).Assembly;
  20.         // and a reference to the type of the EmailAliasRecord internal struct
  21.         Type emailAliasRecordType = sharePointAssembly.GetType(_emailAliasRecordTypeName);
  23.         this.Alias = emailAliasRecordType.GetPublicInstanceFieldValue("alias", emailAliasRecord) as string;
  24.         this.ListId = (Guid)emailAliasRecordType.GetPublicInstanceFieldValue("listId", emailAliasRecord);
  25.         this.WebId = (Guid)emailAliasRecordType.GetPublicInstanceFieldValue("webId", emailAliasRecord);
  26.         this.SiteId = (Guid)emailAliasRecordType.GetPublicInstanceFieldValue("siteId", emailAliasRecord);
  27.         this.IsValid = (bool)emailAliasRecordType.GetPublicInstancePropertyValue("IsValid", emailAliasRecord);
  28.     }
  30.     public EmailAliasRecord(SqlDataReader reader) : this()
  31.     {
  32.         object emailAliasRecord = null;
  34.         // hack to get the Microsoft.SharPoint assembly
  35.         Assembly sharePointAssembly = typeof(SPWeb).Assembly;
  36.         // and a reference to the type of the EmailAliasRecord internal struct
  37.         Type emailAliasRecordType = sharePointAssembly.GetType(_emailAliasRecordTypeName);
  39.         emailAliasRecord = sharePointAssembly.CreateInstance(_emailAliasRecordTypeName);
  41.         if (emailAliasRecord != null)
  42.         {
  43.             emailAliasRecordType.SetPublicInstanceFieldValue("alias", reader.GetString(0), emailAliasRecord);
  44.             emailAliasRecordType.SetPublicInstanceFieldValue("siteId", reader.GetGuid(1), emailAliasRecord);
  45.             emailAliasRecordType.SetPublicInstanceFieldValue("webId", reader.GetGuid(2), emailAliasRecord);
  46.             emailAliasRecordType.SetPublicInstanceFieldValue("listId", reader.GetGuid(3), emailAliasRecord);
  48.             InitFields(emailAliasRecord);
  49.         }
  50.     }
  52.     public override string ToString()
  53.     {
  54.         StringBuilder sb = new StringBuilder();
  56.         using (SPSite site = new SPSite(this.SiteId))
  57.         {
  58.             using (SPWeb web = site.OpenWeb(this.WebId))
  59.             {
  60.                 SPList list = web.Lists[this.ListId];
  62.                 sb.AppendFormat("{0} e-mail alias '{1}' found\r\n", this.IsValid ? "Valid" : "Invalid", this.Alias);
  64.                 if (this.IsValid)
  65.                 {
  66.                     sb.Append("Mapped to:'\r\n");
  67.                     sb.AppendFormat("  Web title: {0}\r\n", web.Title);
  68.                     sb.AppendFormat("  Web URL: {0}\r\n", web.Url);
  69.                     sb.AppendFormat("  List title: {0}\r\n", list.Title);
  70.                 }
  71.             }
  72.         }
  74.         return sb.ToString();
  75.     }
  77. }

Using the code above, it is rather straightforward to call the private GetEmailAliasRecordFromDatabase method of the internal SPEmailMap class (namespace and assembly as above):

  1. private void GetEmailAliasRecordFromDatabase(string mailAlias)
  2. {
  3.     bool found = false;
  5.     string spEmailMapTypeName = "Microsoft.SharePoint.Administration.SPEmailMap";
  6.     // hack to get the Microsoft.SharPoint assembly
  7.     Assembly sharePointAssembly = typeof(SPWeb).Assembly;
  8.     // and a reference to the type of the SPElementProvider internal class
  9.     Type spEmailMapType = sharePointAssembly.GetType(spEmailMapTypeName);
  12.     // spEmailMap will be of type internal class
  13.     // Microsoft.SharePoint.Administration.SPEmailMap
  14.     // defined in Microsoft.SharePoint assembly
  15.     object spEmailMap = sharePointAssembly.CreateInstance(spEmailMapTypeName, false,
  16.         BindingFlags.Public | BindingFlags.Instance, null, null, CultureInfo.InvariantCulture, null);
  18.     if (spEmailMap != null)
  19.     {
  20.         // we call
  21.         // internal EmailAliasRecord GetEmailAliasRecordFromDatabase(string alias)
  22.         MethodInfo mi_GetEmailAliasRecordFromDatabase = spEmailMapType.GetMethod("GetEmailAliasRecordFromDatabase",
  23.                 BindingFlags.NonPublic | BindingFlags.Instance, null,
  24.                 new Type[] { typeof(string) }, null
  25.                 );
  26.         if (mi_GetEmailAliasRecordFromDatabase != null)
  27.         {
  28.             object result = mi_GetEmailAliasRecordFromDatabase.Invoke(spEmailMap,
  29.                 new Object[] { mailAlias });
  31.             EmailAliasRecord ear = new EmailAliasRecord(result);
  33.             if (ear.IsValid)
  34.             {
  35.                 found = true;
  36.                 Console.WriteLine(ear.ToString());
  37.             }
  38.         }
  39.     }
  41.     if (!found)
  42.     {
  43.         Console.WriteLine("Found no valid mapping for e-mail alias '{0}'", mailAlias);
  44.     }
  45. }

This code will output the Title property of the associated SPWeb and SPList objects, as well as the Url property of the SPWeb, thus helping you to find out where you set the specified mail alias.

October 10, 2012

Resolving SharePoint users based on their e-mail address

Filed under: Incoming email, SP 2010 — Tags: , — Peter Holpar @ 20:22

If you send a meeting request to a SharePoint calendar, the default SharePoint infrastructure does not resolve the participants’ mail addresses to meeting attendees (see Issue 2 in my former post).

Last week I was working on a solution for this issue, one of my tasks was to find out the user based on the e-mail address.

Trying to solve this problem we might apply a few alternative approaches, but most of them have a significant drawback:

  • Use an SPUserCollection instance, like the AllUsers property of the SPWeb object, then call its GetByEmail method. Unfortunately, if there is no matching user in the collection, this method throws an exception, and most importantly, if the user has access to the site through a group membership (that is a rather common case), it simply does not work.
  • Use the user profile service to look up the user. Despite being not the best performing way (especially if there is a large number of user profiles), this approach might at least work. To make things a bit faster, instead of iterating through all profiles we can try to crawl the user profile properties and use Search API for the lookup. However, we need a solution that is accessible in the case of SharePoint Foundation as well, so it was a dead-end in our case.
  • Since the People Picker in SharePoint already has a similar functionality, I’ve analyzed its implementation using Reflector, and then came up with the following – quite simple and fast – solution:

SPPrincipalInfo pi = SPUtility.ResolveWindowsPrincipal(site.WebApplication, mailAddress, SPPrincipalType.User, true);
if ((pi != null) && (!string.IsNullOrEmpty(pi.LoginName)))
    SPUser user = web.EnsureUser(pi.LoginName);
    Console.WriteLine("User LoginName: {0}; Name: {1}", user.LoginName, user.Name);

October 9, 2012

Accessing meeting request properties from mail event receivers

Filed under: Calendar, Incoming email, SP 2010 — Tags: , , — Peter Holpar @ 23:34

Assume you have an incoming mail enabled SharePoint calendar, you send meeting requests to the calendar and need to access their properties (like the start or end time of the meeting)  from the code of your SPEmailEventReceiver.

The incoming meeting requests contain the information about the meeting as a separate attachment in the iCalendar format.

You can read about the iCalendar standard on Wikipedia, like

“By default, iCalendar uses the UTF-8 character set.”


“iCalendar data has the MIME content type text/calendar.”

Although for parsing this content you can’t find the necessary methods in the core SharePoint object model, SharePoint uses the CalendarReader (and related) objects located in the Microsoft.Internal.Mime assembly from GAC when processing incoming meeting requests.

The following code illustrates, how to get a property by its name (see the Wikipedia article for possible values) from the SPEmailMessage object received in your event handler.

  1. private string GetCalendarProp(SPEmailMessage mail, string propName)
  2.         {
  3.             string propValue = null;
  5.             foreach (SPEmailAttachment att in mail.Attachments)
  6.             {
  7.                 if (att.ContentType == "text/calendar")
  8.                 {
  9.                     using (Stream stream = att.ContentStream)
  10.                     {
  11.                         CalendarReader cr = new CalendarReader(stream, "utf-8", CalendarComplianceMode.Loose);
  12.                         while (cr.ReadNextComponent() && string.IsNullOrEmpty(propValue))
  13.                         {
  14.                             if (cr.ComponentId == ComponentId.VEvent)
  15.                             {
  16.                                 CalendarPropertyReader pr = cr.PropertyReader;
  18.                                 while (pr.ReadNextProperty() && (string.IsNullOrEmpty(propValue)))
  19.                                 {
  20.                                     if (pr.Name == propName)
  21.                                     {
  22.                                         propValue = pr.ReadValueAsString();
  23.                                     }
  24.                                 }
  25.                             }
  26.                         }
  27.                     }
  28.                     break;
  29.                 }
  30.             }
  32.             return propValue;
  33.         }

Because it is rather cumbersome to test event receivers, you can temporarily stop the SPTimer service (don’t forget to restart it later!), send your meeting request to SharePoint, and when the mail arrives to the Drop folder of SMTP, copy a backup to another folder. Next, you can use a console application with a method like this one below to read the SPEmailMessage directly from the file:

  1. private void SPEmailMessageTest()
  2. {
  3.     using (FileStream fs = new FileStream(@"C:\temp\appointment – 159d46ff01cda09100000003.eml", FileMode.Open))
  4.     {
  5.         SPEmailMessage mail = new SPEmailMessage(fs, "test");
  6.         string uid = GetCalendarProp(mail, "UID");
  7.     }
  9. }

The sample method above reads the UID property of the meeting that is an internal ID to look up and update former corresponding calendar items.

Checking programmatically if an e-mail address is already in use for incoming mail

Filed under: Incoming email, Reflection, SP 2010 — Tags: , , — Peter Holpar @ 21:53

Recently I work quite a lot with the SharePoint calendar and the incoming mail feature of SharePoint. During my experiments I found an interesting problem, namely how one can check from code if a specific e-mail address is already configured for any of the lists.

Note: Since the domain-part of the address – that is the part after the @ sign – is fixed as part of the System Settings at the Central Administration, we can only set the e-mail alias – the part before the @ sign – for the lists.


Without this capability, the only option is the trial and error method that means we try to assign the mail address from code, and call the Update method of SPList as illustrated below:

SPList list = web.Lists[listName];
list.EmailAlias = "MailAlias"; // the part of the e-mail address before the @ sign

In the case the alias is already reserved for another list, we receive an SPException (Unable to assign this e-mail address to the list, because the address is in use.) that we could optionally handle using a try/catch block.

As you may know using this kind of error handling structure for checking existing items does not perform well, so if there is a great chance for conflicting mail addresses, it would be far better (and faster) to make the check without exceptions.

Let’s see what happens when we try to save the list configuration either from the UI or by custom code.

First, the Update(bool bFromMigration) method of the SPList is called, that calls the AssignAlias(string alias, SPList list) method of the SPEmailMap class. It validates the format of the alias via the static ValidateAlias(string alias) method, and then check the existence of a list having this alias through the CanGetListFromDatabase(string alias) method. This method uses another method GetEmailAliasRecordFromDatabase(string alias) that access the SharePoint database and returns list information in the form of an EmailAliasRecord object.

Note: the e-mail addresses used for this kind of check are stored in the EmailEnabledLists table (with fields Alias, Deleted, SiteId, WebId, ListId) of the configuration database SharePoint. The stored procedure used to get the list data based on the alias is the proc_getEmailEnabledListByAlias.

In the following code, I illustrate how to call the CanGetListFromDatabase method to check the existence of an alias.

Note: the code below uses non-public API calls and so it is not a supported approach. Use this sample at you own risk and preferably only in test environments.

  1. private bool IsAliasInUse(string mailAlias)
  2. {
  3.     bool result = false;
  5.     string spEmailMapTypeName = "Microsoft.SharePoint.Administration.SPEmailMap";
  6.     // hack to get the Microsoft.SharPoint assembly
  7.     Assembly sharePointAssembly = typeof(SPWeb).Assembly;
  8.     // and a reference to the type of the SPEmailMap internal class
  9.     Type spEmailMapType = sharePointAssembly.GetType(spEmailMapTypeName);
  12.     // spEmailMap will be of type internal class
  13.     // Microsoft.SharePoint.Administration.SPEmailMap
  14.     // defined in Microsoft.SharePoint assembly
  15.     object spEmailMap = sharePointAssembly.CreateInstance(spEmailMapTypeName, false,
  16.         BindingFlags.Public | BindingFlags.Instance, null, null, CultureInfo.InvariantCulture, null);
  18.     if (spEmailMap != null)
  19.     {
  20.         MethodInfo mi_CanGetListFromDatabase = spEmailMapType.GetMethod("CanGetListFromDatabase",
  21.                 BindingFlags.NonPublic | BindingFlags.Instance, null,
  22.                 new Type[] { typeof(string) }, null
  23.                 );
  24.         if (mi_CanGetListFromDatabase != null)
  25.         {
  26.             // result is bool
  27.             result = (bool)mi_CanGetListFromDatabase.Invoke(spEmailMap,
  28.                 new Object[] { mailAlias });
  30.         }
  31.     }
  33.     return result;
  34. }

September 15, 2012

Issues processing incoming e-mails to a calendar

Filed under: Calendar, Incoming email, SP 2010 — Tags: , , — Peter Holpar @ 21:16

These are rather trivial issues, but might be useful for others, so I post them here.

Issue 1:

Recently we created an incoming e-mail enabled calendar to track events at a company. One of the users complained because the mails sent to the specified e-mail address did not appear in the calendar, even after the list was set to accept all mails regardless of the security configured. Other users had no such problem.

I’ve checked the SMTP log first, and found there info about the mails but no errors at all. However, in the SharePoint log file I found this one:

Warning –  An error occurred while processing the incoming e-mail file C:\Inetpub\mailroot\Drop\{id of the e-mail}.eml. The error was: Value cannot be null. Parameter name: stream.

Although it is not easy to identify the source of the problem from this message, the reason for this warning turned out to be that the user was sending simple mails an no meeting requests to the list.

Issue 2:

Meeting requests were sent to a group calendar, but none of them was visible on the web UI, when the users clicked on the name of the calendar list on the Quick Launch.

No error in the SMTP nor in the SharePoint logs.

Reason: the items are visible in the non-default views (like Current Events or All Events). The default Calendar view shows only items where the value of the Attendees field is not empty, however, this field is not populated based on the incoming meeting request.

March 12, 2012

Solving the external data access security issue in the case of the OWS process

In the recent two posts I wrote about a security related problem we found when tried to access an external data source from the SharePoint 2010 Timer service process.

After presenting a workaround in the first part, last time I promised a real solution for the issue.

The key to the solution seems to be the information one can found on this MSDN page.

As it states, “The user security token is not available in every context. Without the security token, PassThrough security will not work.

As you may know, PassThrough is the standard and usually recommended authentication method to an external data source, but in this case we should switch to an alternative one.

The trivial solution would be to use the RevertToSelf authentication. Since this type of authentication is not recommended in a production environment, it is disabled by default. Before using it, you should enable it, for example with the help of PowerShell (see an example here).

After you enabled RevertToSelf, you can find the equivalent BDC Identity option in the list of the available authentication modes:


(To access the settings above, you should select the External Systems view at the administration of the Business Data Connectivity Service, then click the name of the external system you would like to manage, and then click the name of the external system instance.)

After you selected BDC Identity authentication mode, you can use this code to access the external system:

  1. using (SPSite site = new SPSite("http://sp2010&quot;))
  2. {
  3.     using (SPWeb web = site.OpenWeb())
  4.     {
  5.         Guid siteId = site.ID;
  6.         Guid webId = web.ID;
  8.         SPSecurity.RunWithElevatedPrivileges(delegate()
  9.         {
  10.             using (SPSite siteImp = new SPSite(siteId))
  11.             {
  12.                 // access external list here
  13.             }
  14.         });
  15.     }
  16. }

The other, and recommended option for authentication is to use the Secure Store Service (SSS).

Note: Secure Store Service is not included in SharePoint Foundation 2010, so this option is unfortunately limited for SharePoint Server 2010 Standard and Enterprise versions.

Create a target application in SSS,


then set the credentials of an account with permissions for the external system.

Next, configure your external system to Impersonate Windows Identity, set the name of the Secure Store Target Application Id as created in the previous step, and set  Secure Store Implementation as Microsoft.Office.SecureStoreService.Server.SecureStoreProvider, Microsoft.Office.SecureStoreService, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c.


After you configured your external system instance as described above, you can use this code to access the external data from the timer process job / event receiver:

  1. using (SPServiceContextScope scope = new SPServiceContextScope(SPServiceContext.GetContext(web.Site)))
  2. {
  3.     // access external list within this block
  4. }

Important to note, that based on the MSDN article mentioned at the beginning of this post, workflows and sandboxed solutions might suffer from the same security problem, so if you have such issues accessing an external data source, the solutions described above might help you in these cases as well.

Older Posts »

Create a free website or blog at