Second Life of a Hungarian SharePoint Geek

May 16, 2016

How to Copy a Document between Folders of a SharePoint Document Library using VBA and REST

Filed under: REST, SP 2013, VBA — Tags: , , — Peter Holpar @ 15:59

One of our customers wanted to copy Office documents (.docx, .xlsx, .xlsm) from a folder of a SharePoint document library into another one using Excel macros. The Excel document, that contains the macro, is located in the root of the document library.

For some mysterious reason, that we could not have really identified, the files were created having a size of 0 bytes at the target location for these types of files, when they were copied using the “classical” methods (more on these methods a bit later), although other file types, like text or image files could be copied without problem.

So what are those classical methods, most of them available already in other blogs or forum threads?

Version 1

Copy the files using the FileSystemObject.

You should add a reference to the Windows Script Host Obejct Model library in VBA.

Note: We convert the URL of the document library into an UNC form in the ConvertPath method. That means, it converts a URL like http://YourSharePoint/DocLib into \\YourSharePoint\DocLib. However, if you have configured HTTPS for your SharePoint, you need to convert the URL into this form: \\YourSharePoint@SSL\DavWWWRoot\DocLib. In this case, you should either extend the ConvertPath method, or simply use a fix path in your code as a quick and dirty solution.

Function ConvertPath(path) As String
  ConvertPath = Replace(path, " ", "%20")
  ConvertPath = Replace(ConvertPath, "/", "\")
  ConvertPath = Replace(ConvertPath, "http:", "")
End Function

Private Sub CopyFiles1()
  Dim sDocPath As String
  Dim sDocPathConv As String
  Dim sFileName As String
  Dim sTargetPath As String
  Dim sSourcePath As String
  Dim fso As FileSystemObject

  Set fso = New FileSystemObject ‘ CreateObject("Scripting.FileSystemObject")
  sDocPath = ThisWorkbook.Path
  sFileName = "WorkBook.xlsx"

  sDocPathConv = ConvertPath(sDocPath)

  sSourcePath = sDocPathConv  & "\Folder1\" & sFileName
  Debug.Print "Source: " & sSourcePath
       
  sTargetPath = sDocPathConv  & "\Folder2\" & sFileName
  Debug.Print "Target: " & sTargetPath

  fso.CopyFile sSourcePath, sTargetPath, True

End Sub

Version 2

Copy the files using SharePoint document library as mapped drive using the FileSystemObject.

In addition to the Windows Script Host Obejct Model library, you need an additional reference to the WSHControllerLibrary as well.

See a similar sample here.

The MapNetworkDrive method seems to handle the conversion of the SharePoint doc. lib. URL into an UNC form, but you might still need to invoke the conversion method if you receive this error on mapping the drive:

800704DC – The operation being requested was not performed because the user has not been authenticated

Private Sub CopyFiles2()
  Dim sDocPath As String
  Dim sFileName As String
  Dim sTargetPath As String
  Dim sSourcePath As String
  Dim sDriveLetter As String
  Dim fso As FileSystemObject
  Dim net As WshNetwork

  ‘ drive letter should be available (not mapped to a share already, to avoid error ‘80070055 – The local device is already in use’)
  sDriveLetter = "S:"
  sFileName = "WorkBook.xlsx"
   
  Set fso = New FileSystemObject ‘ CreateObject("Scripting.FileSystemObject")
  sDocPath = ThisWorkbook.Path

  ‘sDocPath = ConvertPath(sDocPath)

  Set net = New WshNetwork ‘ CreateObject("WScript.Network")
  Debug.Print "Path to map: " & sDocPath
  net.MapNetworkDrive sDriveLetter, sDocPath

  sSourcePath = sDriveLetter & "\Folder1\" & sFileName
  Debug.Print "Source: " & sSourcePath

  sTargetPath = sDriveLetter "\Folder2\" & sFileName
  Debug.Print "Target: " & sTargetPath

  fso.CopyFile sSourcePath, sTargetPath, True

  net.RemoveNetworkDrive sDriveLetter

  Set net = Nothing
  Set fso = Nothing

End Sub

Version 3

We could have downloaded the file, and upload it via web service calls, but I felt this second part simply far too complex.

Version 4

We have used a method to create temporary folder names:

Private Declare Function GetTempPath Lib "kernel32" Alias "GetTempPathA" (ByVal nBufferLength As Long, ByVal lpBuffer As String) As Long

Private Declare Function GetTempFileName Lib "kernel32" Alias "GetTempFileNameA" _
  (ByVal lpszPath As String, _
  ByVal lpPrefixString As String, _
  ByVal wUnique As Long, _
  ByVal lpTempFileName As String) As Long

Public Function Get_Temp_File_Name( _
  Optional sPrefix As String = "VBA", _
  Optional sExtension As String = "") As String

  Dim sTmpPath As String * 512
  Dim sTmpName As String * 576
  Dim nRet As Long
  Dim F As String

  nRet = GetTempPath(512, sTmpPath)
  If (nRet > 0 And nRet < 512) Then
    nRet = GetTempFileName(sTmpPath, sPrefix, 0, sTmpName)
    If nRet <> 0 Then F = Left$(sTmpName, InStr(sTmpName, vbNullChar) – 1)
    If sExtension > "" Then
      Kill F
      If Right(F, 4) = ".tmp" Then F = Left(F, Len(F) – 4)
      F = F & sExtension
    End If

    Get_Temp_File_Name = F
  End If

End Function

Then, instead of copying directly between the document library folders in Version 2 and 3, we copied the file first from source folder to the local temporary file in the file system, then from the temporary file to the target folder.

That means, instead of

fso.CopyFile sSourcePath, sTargetPath, True

we used this:

Dim tempPath As String

tempPath = Get_Temp_File_Name
Debug.Print "Temp path: " & tempPath

fso.CopyFile sSourcePath, tempPath, True
fso.CopyFile tempPath, sTargetPath, True
fso.DeleteFile tempPath

It did not help to remedy our problem with the empty Office files created during the copy operation.

Version 5

This is our last sample code, and it is the one that works at the customer without problem with the file size / content. In this case we utilize the getfilebyserverrelativeurl and copyto methods of the files and folders REST API.

Using this method has a further benefit, that – depending on the file size – might be even a significant one. In contrast with the other methods described earlier, this one does not download / upload the file content. It sends only a command to the server to copy the file, after we perform the authentication in the GetDigest method.

For this example to work, you need a reference to the Microsoft XML, v6.0 library in VBA.

Private Function GetDigest(url As String)
  Dim http As MSXML2.XMLHTTP

  Set http = New MSXML2.XMLHTTP

  http.Open "POST", url + "/_api/contextinfo", False
  http.setRequestHeader "ContentType", "application/json;odata=verbose"
  http.send ""

  GetDigest = http.responseXML.SelectSingleNode("//d:FormDigestValue").nodeTypedValue
 
  Set http = Nothing

End Function

Private Sub CopyFiles3()
  Dim webAppUrl As String
  Dim serverRelUrlOfSite As String
  Dim siteUrl As String
  Dim docLibName As String
  Dim serverRelUrlOfDocLib As String
  Dim sourcePath As String
  Dim targetPath As String

  Dim http As MSXML2.XMLHTTP
  Dim digest As String
  Dim url As String

  webAppUrl = "http://YourSharePoint&quot;
  serverRelUrlOfSite = "/subsite1/subsite1.2"
  docLibName = "YourDocLib"
 
sFileName = "WorkBook.xlsx"

  siteUrl = webAppUrl & serverRelUrlOfSite
  serverRelUrlOfDocLib = serverRelUrlOfSite & "/" & docLibName

  sourcePath = "/Folder1/" & sFileName
  Debug.Print "Source: " & sourcePath

  targetPath = "/Folder2/" & sFileName"
  Debug.Print "Target: " & targetPath

  ‘ get the authentication digest
  digest = GetDigest(siteUrl)
  Set http = New MSXML2.XMLHTTP

  url = siteUrl & "/_api/web/getfilebyserverrelativeurl(‘" & serverRelUrlOfDocLib & sourcePath & "’)/copyto(strnewurl=’" & serverRelUrlOfDocLib & targetPath & "’,boverwrite=true)"

  http.Open "POST", url, False
  http.setRequestHeader "X-RequestDigest", digest

  http.send ""
  
  Set http = Nothing

End Sub

If you need to move the files instead of copying, you should simply use the MoveTo method instead of the CopyTo method.

Note: As you see, this code does not contain any error handling, so please extend it if you would like to use it in production.

PWA Settings Link is not Available on the Central Administration Web Site

Filed under: PS 2013 — Tags: — Peter Holpar @ 15:57

After un-provisioning and re-provisioning a PWA instance in a SharePoint farm, we found, that the PWA Settings link is no more available at the General Application Setting on the Central Administration site.

image

When trying to access the PWA Settings pages, we had two types of errors:

No Project Web App instances found. Create at least one PWA instance before accessing this page.

In the ULS logs:

Application error when access /_layouts/15/pwa/Admin/Admin.aspx, Error=No Project Web App instances found. Create at least one PWA instance before accessing this page.   at Microsoft.Office.Project.PWA.PJBasePage..ctor(String pwaUrl)     at Microsoft.Office.Project.PWA.PJWebPage..ctor(String url)     at __ASP.FastObjectFactory_app_web_admin_aspx_ad835a1b_9lzjizur.Create_ASP__layouts_15_pwa_admin_admin_aspx()     at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualPath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp)     at System.Web.UI.PageHandlerFactory.GetHandlerHelper(HttpContext context, String requestType, VirtualPath virtualPath, String physicalPath)     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.Ht…
…tpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Microsoft.SharePoint.SPException: No Project Web App instances found. Create at least one PWA instance before accessing this page.    at Microsoft.Office.Project.PWA.PJBasePage..ctor(String pwaUrl)     at Microsoft.Office.Project.PWA.PJWebPage..ctor(String url)     at __ASP.FastObjectFactory_app_web_admin_aspx_ad835a1b_9lzjizur.Create_ASP__layouts_15_pwa_admin_admin_aspx()     at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualPath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp)     at System.Web.UI.PageHandlerFactory.GetHandlerHelper(HttpContext context, String requestType, VirtualPath virtualPath, String physicalPath)     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute…
…()     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Getting Error Message for Exception Microsoft.SharePoint.SPException: No Project Web App instances found. Create at least one PWA instance before accessing this page.     at Microsoft.Office.Project.PWA.PJBasePage..ctor(String pwaUrl)     at Microsoft.Office.Project.PWA.PJWebPage..ctor(String url)     at __ASP.FastObjectFactory_app_web_admin_aspx_ad835a1b_9lzjizur.Create_ASP__layouts_15_pwa_admin_admin_aspx()     at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualPath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp)     at System.Web.UI.PageHandlerFactory.GetHandlerHelper(HttpContext context, String requestType, VirtualPath virtualPath, String physicalPath)     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.H…
…ttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

and

File Not Found

In the ULS logs:

Application error when access /_layouts/15/pwa/Admin/Admin.aspx, Error=The site with the id 5584122d-2bdd-48b5-bd15-7885712b9892 could not be found.   at Microsoft.SharePoint.SPSite..ctor(Guid id, SPFarm farm, SPUrlZone zone, SPUserToken userToken)     at Microsoft.Office.Project.PWA.PJContext.GetContextForCentralAdmin(Boolean isWebServiceCall, Guid siteID)     at Microsoft.Office.Project.PWA.PJBasePage..ctor(String pwaUrl)     at Microsoft.Office.Project.PWA.PJWebPage..ctor(String url)     at __ASP.FastObjectFactory_app_web_admin_aspx_ad835a1b_9lzjizur.Create_ASP__layouts_15_pwa_admin_admin_aspx()     at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualPath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp)     at System.Web.UI.PageHand…
…lerFactory.GetHandlerHelper(HttpContext context, String requestType, VirtualPath virtualPath, String physicalPath)     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
System.IO.FileNotFoundException: The site with the id 5584122d-2bdd-48b5-bd15-7885712b9892 could not be found.    at Microsoft.SharePoint.SPSite..ctor(Guid id, SPFarm farm, SPUrlZone zone, SPUserToken userToken)     at Microsoft.Office.Project.PWA.PJContext.GetContextForCentralAdmin(Boolean isWebServiceCall, Guid siteID)     at Microsoft.Office.Project.PWA.PJBasePage..ctor(String pwaUrl)     at Microsoft.Office.Project.PWA.PJWebPage..ctor(String url)     at __ASP.FastObjectFactory_app_web_admin_aspx_ad835a1b_9lzjizur.Create_ASP__layouts_15_pwa_admin_admin_aspx()     at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualPath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp)     at System.Web.UI.PageHandlerFactory.GetHandlerHelper(HttpConte…
…xt context, String requestType, VirtualPath virtualPath, String physicalPath)     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
Getting Error Message for Exception System.Web.HttpException (0x80004005): Exception of type ‘System.Web.HttpException’ was thrown. —> System.IO.FileNotFoundException: The site with the id 5584122d-2bdd-48b5-bd15-7885712b9892 could not be found.     at Microsoft.SharePoint.SPSite..ctor(Guid id, SPFarm farm, SPUrlZone zone, SPUserToken userToken)     at Microsoft.Office.Project.PWA.PJContext.GetContextForCentralAdmin(Boolean isWebServiceCall, Guid siteID)     at Microsoft.Office.Project.PWA.PJBasePage..ctor(String pwaUrl)     at Microsoft.Office.Project.PWA.PJWebPage..ctor(String url)     at __ASP.FastObjectFactory_app_web_admin_aspx_ad835a1b_9lzjizur.Create_ASP__layouts_15_pwa_admin_admin_aspx()     at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualP…
…ath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp)     at System.Web.UI.PageHandlerFactory.GetHandlerHelper(HttpContext context, String requestType, VirtualPath virtualPath, String physicalPath)     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.MaterializeHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

PowerShell cmdlets have not found any site with the given Id (5584122d-2bdd-48b5-bd15-7885712b9892) as well, thus I made a query against the SharePoint configuration database to find out more details:

SELECT [Id],
    [ClassId],
    [ParentId],
    [Name],
    [Status],
    [Version],
    [Properties]
FROM
    [SharePointConfigDB].[dbo].[Objects]
WHERE
    [Properties] LIKE ‘%5584122d-2bdd-48b5-bd15-7885712b9892%’

Note: You should replace the database name and the Guid in the query, if you happen to have the same issue, and would like to follow my steps.

This query returned a configuration object of type Microsoft.Office.Project.Server.Administration.ProjectSite, having a Name = 5584122d-2bdd-48b5-bd15-7885712b9892, ClassId = FB30E224-C739-43A7-8BEA-11D170824A6A, and an Id = 13B1DAD3-B005-46D9-B712-CDF7CC2FEAFF.

I was able to delete the orphaned object via the command:

STSADM -o deleteconfigurationobject -id 13B1DAD3-B005-46D9-B712-CDF7CC2FEAFF

This deletion solved our problem.

Note: There is no guarantee, that this solution works for you as well. Don’t forget to create a backup of your configuration database before you performs this kind of changes!

“Project Professional needs to be installed to open the project. Install Project and try again.” Warning, When Trying to Open Project in Microsoft Project

Filed under: PS 2013 — Tags: — Peter Holpar @ 15:56

Recently a few users complained that they receive this warning in Internet Explorer (version 11) every time they try to open a project from PWA using Microsoft Project (for example, by clicking the Gantt-icon left to the project name in the Project Center):

Action Not Available
Project Professional needs to be installed to open the project. Install Project and try again.

Despite of the warning, the project was opened without other problems, but the users found this behavior disturbing. Other users with the same browser version did not have any problem opening projects in the same way.

I found this and this related entries on the web, but none of them helped to solve the issue in my case.

As it turned out, a JavaScript component called ProjectServer.Imported.ActiveXTaskLauncher (defined in 15\TEMPLATE\LAYOUTS\INC\PWA\LIBRARY\ps.core.js) is responsible for starting Project Professional from the browser.

The way the external application should be started depends on the browser version, in our case this code block performs the action:

var h=setTimeout(function(){
    a:;
    window.removeEventListener("blur",c);
    d()
},1e3);
function c(){
    a:;
    clearTimeout(h);
    window.removeEventListener("blur",c)
}
var i=window.addEventListener("blur",c);
window.location.href=b

where function d() contains displaying the warning:

function d(){
        a:;
        PJMessageBox(PJMessageType.Error,PWA.Res.GENERAL_ERROR_ACTION_NOT_AVAILABLE,PWA.Res.WEBPARTS_PROJECTCENTER_PROJPRO_REQUIRED,SP.UI.DialogResult.OK,null)
    }

Although it is not obvious at the first sight, what the first code block does, it starts a timer that should display the warning in 1e3 (that is 103 in JavaScript notation) milliseconds ( = 1 sec.) by calling function d(), unless this action is cancelled by function c() if a focus loss of the browser is detected via the event listener subscribed for the event blur.

The value of the b variable, that is used to set the value of the window.location.href in the last line of the first cod block above:

ms-project:osp|u|https://YourProjectServer/PWA/|g|50e73545-3ee3-489c-963d-0e0f06050737|p|<>\\YourProject|r|0

where the Guid is the ID of the PWA site collection. This value is passed to the ProjectServer.Imported.ActiveXTaskLauncher constructor by the GetActiveXTaskLauncher method of the PJ.ProjectDrilldownSatelliteImpl (15\TEMPLATE\LAYOUTS\INC\PWA\LIBRARY\ProjectDrilldownSatellite.js).

In our case the issue was performance-related: Project Professional was simply not started within the 1 second limit. The quick and dirty “solution” was to increase the time-out value to 1e4 (that is 10 seconds) by editing the ps.core.js.

Note: this workaround is generally considered as a bad practice, as it contains editing of  standard Project Server files. The patch state of the farm is 2016 March CU. I hope the 2016 April CU provides a solution for the problem, as its description at the Project Server 2013 Cumulative Updates page contains this one in the list of fixed issues:

Even though you have already installed Project 2013, you may still receive the following error message from PWA:
Action Not Available.
Project Professional needs to be installed to open the project. Install Project and try again.

Project Server Displays Incorrect Effective Rights for Resources

Filed under: Bugs, PS 2013, Security — Tags: , , — Peter Holpar @ 15:55

We observed the following – in my opinion buggy – behavior in case of Project Server 2013 (patch state: 2016 March CU):

In our project web sites we have a web page that should display the name of the project and the title of the project owner. The name is displayed using client-side technologies, that means JavaScript and the Project Server JavaScript object model. See the code snippets below. Note, that these are parts of an AngularJS applications and cannot be used alone, but only part of the whole application. I show the code only to provide you an overview, about what I’m writing here.

The “business logic” from the controller:

  1. var promise = OurCustomService.getProjInfo($scope);
  2. promise.then(function (pi) {
  3.     var projName = pi.project.get_name();
  4.     // for some users the get_owner() mehtod returns null
  5.     var projManName = pi.project.get_owner().get_title();            
  6. }, function (errorMsg) {
  7.     console.log("Error: " + errorMsg);
  8. });
  9.  
  10.     }, function (errorMsg) {
  11.         console.log("Error: " + errorMsg);
  12.     });
  13. });

…and the service code:

  1. this.getProjInfo = function ($scope) {
  2.     var deferred = $q.defer();
  3.  
  4.     var ctx = new SP.ClientContext.get_current();
  5.  
  6.     var projContext = PS.ProjectContext.get_current();
  7.     projContext.set_isPageUrl(ctx.get_isPageUrl);
  8.     var proj = projContext.get_projects().getById($scope.projectId);
  9.     projContext.load(proj, "Name", "Owner.Title");
  10.  
  11.     projContext.executeQueryAsync(
  12.         function () {
  13.             deferred.resolve(
  14.                 {
  15.                     project: proj
  16.                 });
  17.         },
  18.         function (sender, args) {
  19.             deferred.reject('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
  20.         }
  21.     );
  22.  
  23.     return deferred.promise;
  24. };

We found that this solution does not work for a lot of our users. Having a look via Internet Explorer (F12) Developer Tools in the code running with their credentials I found that the object returned by the pi.project.get_owner() expression is null, causing an exception as I want to access the get_title() method of this null object. It was obviously a security issue. To be able to access the title of the project owner (that is a resource as well), the user should have the View Enterprise Resource Data category permission in relation to the project owner resource.

When checking the Owner property of the Project via REST (the Guid in the URL is the ID of the given project):

http://YourProjectServer/PWA/_api/ProjectServer/Projects(‘1EF03FA9-2F7A-E411-80D4-005056B47337&#8217;)/Owner

the users having the problem received null as result, however, other users having more permissions (including the required one) received the full info of the project owner as expected.

Similarly, we have checked the resources available for the user via the REST query:

http://YourProjectServer/PWA/_api/ProjectServer/EnterpriseResources

The result for the “problematic” users did not contained the resource that is the project owner, however for the other users (the ones who had no problem with the AngularJS application mentioned above) the result included this resource as well.

No problem, it sounds OK up to this point.

However, when we selected any of  “problematic” these users on the Manage Users page in PWA Settings, clicked Check Effective Rights, change the Permission Type to Category Permission – Resource, and selected the selected the project owner (the one, the user has in practice no permission at all) in the Security Object – Resource list, the report shows, that the user has View Enterprise Resource Data permission via a group (let’s say All Users) and a category (let’s say My Project Team). Then we clicked other resources in the Security Object – Resource list as well, and found, that based on the report, the user should have View Enterprise Resource Data permission to almost all of these resources either, although based on the REST query above (http://YourProjectServer/PWA/_api/ProjectServer/EnterpriseResources) he has permission only a very few of them.

That is pretty strange. The users are really member of the All Users group, and the My Project Team category is really assigned to the All Users group.

The resources affected by the My Project Team category are selected by the “They are members of a Project Team on a project owned by the User” rule:

image

Members of the All Users group have View Enterprise Resource Data permission on resources included in the My Project Team category:

image

The resources displayed by the Effective Rights page as ones the “problematic” users have permission to are however no team members of the users at all!

How is it possible? In this post I don’t want to bore you with very deep technical details (I plan to post these details in a follow-up post later), the most important facts are, that the objects and stored procedures used to check the permissions when you want to access a resource differ from the ones used to display the effective rights.

For example, when checking the “They are members of a Project Team on a project owned by the User” rule, the pub.MSP_WEB_FN_SEC_ResourcesInCategoryRule3 table-valued function is used, where @res_uid parameter is the resource ID of the current user. It should return the ID of all of the resources that are affected by this category rule:

SELECT RES_UID AS OBJ_UID
FROM pub.MSP_ASSIGNMENTS
WHERE WRES_UID_MANAGER = @res_uid
  AND WASSN_DELETED_IN_PROJ = 0

UNION

SELECT PR.RES_UID AS OBJ_UID
FROM pub.MSP_PROJECTS P
INNER JOIN pub.MSP_PROJECT_RESOURCES PR ON PR.PROJ_UID = P.PROJ_UID
WHERE P.WRES_UID = @res_uid

That means, resources returned by the query if the resource that belongs to the current user (the one that wants to access another resource) is either an assignment owner of  a non-deleted project task assignment where the target resource (the one the current user want to access) is the assignment resource (first part of the UNION query), or there is a project that has the current user as project manager and the target resource as project resource (second part of the UNION query). That sounds logically.

On the contrary, when displaying the effective rights, the pub.MSP_WEB_FN_SEC_GetEffectiveCategories_NONCLAIMSCOMPLIANT tabled-value function is called by the pub.MSP_WEB_SP_SEC_ReadUserEffectiveRightsWithCategoryPermissions_NONCLAIMSCOMPLIANT stored procedure. This function uses the following condition to check the “They are members of a Project Team on a project owned by the User” rule, where @res_uid parameter is the resource ID of the current user, and the @wsec_obj_uid parameter is the ID of the target resource. It should insert the value 3 into the temporary @rule_table is the target resource is affected by the category rule:

IF EXISTS (SELECT TOP 1 RES_UID FROM MSP_ASSIGNMENTS WHERE WRES_UID_MANAGER = @res_uid AND RES_UID = @wsec_obj_uid)
    OR EXISTS (SELECT TOP 1 RES_UID FROM MSP_PROJECT_RESOURCES WHERE RES_UID = @wsec_obj_uid)
    OR EXISTS (SELECT TOP 1 WRES_UID as RES_UID FROM MSP_PROJECTS WHERE WRES_UID = @res_uid)
BEGIN
    INSERT INTO @rule_table(WSEC_OBJ_RULE_TYPE) VALUES (3)
END

As far as I see, this condition is wrong. It says that there should be an assignment having the current user as an assignment owner and the target resource as assignment resource (see first part of the UNION in the first SQL query above, differs in checking the WASSN_DELETED_IN_PROJ flag), or there is any project, where the target resource is a resource, or there is any project where the current user is the project manager (compare with the second part of the UNION query above, condition this time is total wrong). It means we may receive a false positive on the Effective Rights page for each resources, that are resources on any project, and for all resources if the current user (the one we check the effective rights for) is a project manager of any project. In fact, we should receive a positive value in all of these cases (as long as there is no explicit deny), it is a false positive, when there is no other, valid positive value via other categories.

I think one should re-arrange the condition like this:

IF EXISTS (SELECT TOP 1 RES_UID FROM MSP_ASSIGNMENTS WHERE WRES_UID_MANAGER = @res_uid AND RES_UID = @wsec_obj_uid AND WASSN_DELETED_IN_PROJ = 0)
    OR EXISTS (SELECT TOP 1 PR.RES_UID FROM MSP_PROJECTS P INNER JOIN MSP_PROJECT_RESOURCES PR ON PR.PROJ_UID = P.PROJ_UID
    WHERE PR.RES_UID = @wsec_obj_uid AND P.WRES_UID = @res_uid)
BEGIN
    INSERT INTO @rule_table(WSEC_OBJ_RULE_TYPE) VALUES (3)
END

or even better, one could simply re-use the logic implemented in the pub.MSP_WEB_FN_SEC_ResourcesInCategoryRule3 table-valued function:

IF EXISTS (SELECT TOP 1 OBJ_UID FROM pub.MSP_WEB_FN_SEC_ResourcesInCategoryRule3(@res_uid) WHERE OBJ_UID = @wsec_obj_uid)
BEGIN
    INSERT INTO @rule_table(WSEC_OBJ_RULE_TYPE) VALUES (3)
END

April 20, 2016

How to Start the Wrong SharePoint Workflow Unintentionally from the UI

Filed under: Bugs, SP 2010 — Tags: , — Peter Holpar @ 22:38

A few weeks ago we wanted to start a specific workflow on a SharePoint list item, that is located in a list that has multiple custom workflows (none of them has a workflow initiation form) associated with it. We performed the action at the very same time with a colleague of mine (having only 4 seconds difference, as it turned out later), independently from each other. After a while, when we checked the status of the item, we found, that two workflows started on the item. The one, we both wanted to start, was started by my colleague, and I’ve started an other workflow. My colleague said I might have clicked the wrong workflow on the workflow.aspx web page, but I was sure I clicked the right one.

What happened?

I was able to reproduce the issue in our test environment by loading the Workflow page in two separate browser tabs, and starting the same workflow in each of them. I was pretty confident, that the developers of the page made the mistake to try to start the selected workflow by its index in the array of available workflows, instead of the Id of the workflow association.To be sure, I’ve checked the source of the workflow.aspx file (located in the LAYOUTS folder, having a lot of in-line code) and the class behind it, the Microsoft.SharePoint.ApplicationPages.WorkflowPage class (located in the Microsoft.SharePoint.ApplicationPages assembly).

Note: If you try to start the last workflow on the page (or as a special case of it, you have only a single associated workflow) twice in two separate browser, you get an error on the second try. In this case you have the following entry (Level: Unexpected) in the ULS logs:

System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.  Parameter name: index    at System.Collections.ArrayList.get_Item(Int32 index)     at Microsoft.SharePoint.ApplicationPages.WorkflowPage.OnLoad(EventArgs e)     at System.Web.UI.Control.LoadRecursive()     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

The OnLoad method of the WorkflowPage class calls the ConstructStartArray method if it is not a page post back. In the ConstructStartArray a new ArrayList is created in the m_alwaStart field, and it is populated by the available workflows, by iterating through the workflow associations on the list, on the content type, and on the web level. In each case, the FCanStartWorkflow method of the base class (WorkflowPageBase) is invoked to ensure the current user has the permission to start the workflow manually (see the SPWorkflowAssociation.PermissionsManual property) and if there is no running instance of this workflow type on the item already (via the FIsRunningWt method of the same class). By the end of the ConstructStartArray method the ArrayList in the m_alwaStart field contains the workflows the user can start on the item. So far so good.

Let’s see how this list is rendered in the in-line code of the workflow.aspx page.

<%
    bool fColumnPosition=true;
    int iwa = 0;
    strImageUrl = "/_layouts/images/availableworkflow.gif";
    foreach (SPWorkflowAssociation wa in m_alwaStart)
    {
        string strIniUrl = GenerateStartWorkflowLink(wa);
        if (strIniUrl == null)
            strIniUrl = "javascript:StartWorkflow(" + Convert.ToString(iwa) + ")";
%>

<%

    iwa++;

%>

As you can see, the parameter used with the StartWorkflow method is really a simple counter, the index of the workflow association in the ArrayList in the m_alwaStart field.

The StartWorkflow JavaScript method simply sets a form value (iwaStart) and posts back the page:

function StartWorkflow(iwa)
    var elIwaStart = document.getElementById("iwaStart");
    elIwaStart.value = iwa;
    theForm.submit();
}

The server side GenerateStartWorkflowLink method of the WorkflowPage class, that you can also see in the inline-code above should display the workflow initiation form for the workflow association, if any exists.

Back to the server side, and let’s see what happens with the value posted back by the StartWorkflow method in the OnLoad method of the WorkflowPage class. If the request is a post back, than it reads the index of the workflow to start, and looks up the workflow by this index from the array of workflow associations in the m_alwaStart field:

int num2 = Convert.ToInt32(base.Request.Form["iwaStart"]);
if (num2 >= 0)
{
    base.StartWorkflow((SPWorkflowAssociation) this.m_alwaStart[num2]);
}

Problem: this array might be not the same, as the one returned on the first page load. If a workflow that precedes the workflow we want to start (or the same workflow) is started in the meantime, the workflow associations are changed (for example, workflows are registered or removed on the web, on the list or on the content type level), or the permissions are changed, it is possible (or even very likely) that the user starts another workflow, not the one he clicked on on the web UI.

Solution: would be to use the Id (of type Guid) of the Microsoft.SharePoint.Workflow.SPWorkflowAssociation instance as the identifier of the item in the array instead of  the index / position in the array.

That would mean in the in-line code, instead of using the iwa counter:

strIniUrl = "javascript:StartWorkflow(" + Convert.ToString(wa.Id) + ")";

and in the OnLoad method, handling the post back as:

Guid waId = Guid.Parse(base.Request.Form["iwaStart"] as string);
base.StartWorkflow(this.m_alwaStart.ToArray().First<SPWorkflowAssociation>(wa => wa.Id == waId));

Note: I could reproduce this buggy behavior in SharePoint 2010 and in SharePoint 2013 using site collections that were not upgraded to the SharePoint 2013 mode. However, as long as I see, “native” SharePoint 2013 sites do suffer from the same kind of problem.

Reusing PSI Proxy Objects from PowerShell

Filed under: PowerShell, PS 2013, PSI — Tags: , , — Peter Holpar @ 22:06

Assume you create a PowerShell script that invokes PSI to perform some actions on Project Server. For example, creating a custom field as described in my former post. You save the script as a .ps1 file and invoke it from the PowerShell shell. Assume it has some parameters and your goal is to invoke it multiple times with various parameter sets. On of the first step in the script is of course the creation in the PSI proxy object, as shown in the original version:

$pwaUrl = "http://YourProjectServer/pwa&quot;
$svcPath = "/_vti_bin/psi/CustomFields.asmx?wsdl"

$svcPSProxy = New-WebServiceProxy -Namespace PSIProxy -Uri ($pwaUrl + $svcPath) -UseDefaultCredential

later in your code you invoke a method on the proxy object:

$svcPSProxy.CreateCustomFields($customFieldDataSet, $false, $true)

On the first run of the script it performs the actions without error, however on the next (and on each later) run it gives you an exception like this:

Cannot convert argument "cfds", with value: "PSIProxy.CustomFieldDataSet", for
"CreateCustomFields" to type "PSIProxy.CustomFieldDataSet": "Cannot convert
the "PSIProxy.CustomFieldDataSet" value of type "PSIProxy.CustomFieldDataSet"
to type "PSIProxy.CustomFieldDataSet"."
At line:1 char:1
+ $svcPSProxy.CreateCustomFields($customFieldDataSet, $false, $true)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodException
    + FullyQualifiedErrorId : MethodArgumentConversionInvalidCastArgument

Do you see the strange error message?

Cannot convert the "PSIProxy.CustomFieldDataSet" value of type "PSIProxy.CustomFieldDataSet" to type "PSIProxy.CustomFieldDataSet".

An object having a specific type cannot be converted to a type having the same type name. Very weird. PowerShell seems to cache the object types created dynamically by the New-WebServiceProxy cmdlet on the first run, and these types seem to be not compatible (at least, in the .NET-sense) with the ones created on the next runs. The single (or at least the most simple) solution seems to be to restart the shell after each run, but it is not very nice, to say the least.

Fortunately, I’ve found a better way in this thread for the “recycling” of the proxy object created on the first execution. Note, that the solution I find there is not the accepted answer as I wrote this post. See the answer from existenz7 on February 07, 2013 1:08 PM.

So I’ve changed the proxy creation part in my script to the form:

If ($global:svcPSProxy -eq $null)
{
  Write-Host "Connecting PSI proxy at $pwaUrl …"
  $global:svcPSProxy = New-WebServiceProxy -Namespace PSIProxy -Uri ($pwaUrl + $svcPath) -UseDefaultCredential
}
Else
{
  Write-Host "Reusing existing PSI proxy"
}

You can invoke the proxy method just like earlier:

$svcPSProxy.CreateCustomFields($customFieldDataSet, $false, $true)

Note: I typically omit this kind of proxy creation from my code posted here on the blog just not to disturb you with details that are not relevant to the problem discussed actually in the post. However, I suggest you to apply the same technique to avoid the type incompatibility issue mentioned above.

Handling PSI Errors in PowerShell Scripts

Filed under: PowerShell, PS 2013, PSI — Tags: , , — Peter Holpar @ 22:03

Recently I work pretty much with PSI calls from PowerShell, to automate such administrative tasks, that are not available in the Project Server Client Object Model, like setting the rules of graphical indicators for the custom fields (see full code here). When using such code, sooner or later you receive an PSI exception, either due to lack of data, an invalid data, or simply because the entity you are working with are not checked out to you.

For example, if you call the CreateCustomFields as shown in the former post:

$svcPSProxy.CreateCustomFields($customFieldDataSet, $false, $true)

you may receive a PSI error that is displayed by default so in PowerShell:

Exception calling "CreateCustomFields" with "3" argument(s):
"ProjectServerError(s) LastError=CustomFieldLowerOrderBitsOutOfRange
Instructions: Pass this into PSClientError constructor to access all error
information"
At line:1 char:1
+ $svcPSProxy.CreateCustomFields($customFieldDataSet, $false, $true)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : SoapException

To display the details of the error, I’ve implemented the error handling similar to the one you find in several PSI-related C# article on MSDN (see this one, for example):

Try {
    $svcPSProxy.CreateCustomFields($customFieldDataSet, $false, $true)
}
Catch [System.Web.Services.Protocols.SoapException] {
    $ex = $_.Exception
    $clientError = New-Object Microsoft.Office.Project.Server.Library.PSClientError($ex)   
    $psiErrors = $clientError.GetAllErrors()
    $psiErrors | % {
        $err = $_
        Write-Host ([int]$err.ErrId) $err.ErrName
        For($i = 0; $i -lt $err.ErrorAttributes.Length; $i++) {
            Write-Host $err.ErrorAttributeNames()[$i] $err.ErrorAttributes[$i];
        }
    }
}

In my case, the specific error information is displayed as error code – error name pairs, like:

11521 CustomFieldMaskDoesNotMatchEntityType
11522 CustomFieldLowerOrderBitsOutOfRange

Note 1: I typically omit this kind of error handling from my code posted here on the blog just not to disturb you with details that are not relevant to the problem discussed actually in the post. It may be OK for you to run the code as it is posted as long as there is no error, however if an exception is thrown, it is good to know how to access the details of the problem. Alternatively, you can use Fiddler to capture the server response, and check the raw XML for the error information.

Note 2: A comprehensive list of PSI error codes is available here, including a short description for each error type. See the Microsoft.Office.Project.Server.Library.PSErrorID enumeration as well.

April 18, 2016

“Cannot find an SPSite object” Error From PowerShell

Filed under: PowerShell, SP 2013 — Tags: , — Peter Holpar @ 22:41

Last week I learned something new about a well known error message. We’ve tried to access a SharePoint site collection / site via the Get-SPSite / Get-SPWeb cmdlets in a test environment, but got the errors:

Get-SPWeb : Cannot find an SPSite object that contains the following Id or Url: http://YourSharePoint

and

Get-SPSite : Cannot find an SPSite object that contains the following Id or Url: http://YourSharePoint

In my practice this means either you mistyped the URL (what the message really suggests), using the wrong protocol (HTTP instead of  HTTPS or the opposite one), or your user has not the required permission (like membership in the the SharePoint_Shell_Access role, you can add via the Add-SPShellAdmin cmdlet) in the content database. In this case the URL was right, and I have db_owner permissions on the DBs, so I had to look for another reason.

Calling the Get-SPWebApplication cmdlet returned the URL of  every web applications in the farm, however when I tried to list the site collections in any (!) of these web applications, like this:

$wa = Get-SPWebApplication http://YourSharePoint
$wa.Sites

it gave me this error on the screen:

An error occurred while enumerating through a collection: The HTTP service
located at
http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc is
unavailable.  This could be because the service is too busy or because no
endpoint was found listening at the specified address. Please ensure that the
address is correct and try accessing the service again later..

Similarly, by checking the stack trace (via the value of  $StackTrace) after invoking the Get-SPSite / Get-SPWeb cmdlets, I received an other evidence of the problem with the communication with token service:

   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.Http
ChannelRequest.WaitForReply(TimeSpan timeout)

The corresponding details from ULS logs:

Entering BeginProcessing Method of Get-SPSite.
Leaving BeginProcessing Method of Get-SPSite.
Entering ProcessRecord Method of Get-SPSite.
SecurityTokenServiceSendRequest: RemoteAddress: ‘http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc&#8217; Channel: ‘Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustChannelContract’ Action: ‘http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue&#8217; MessageId: ‘urn:uuid:b7dfe754-fdd4-4177-bec6-f8992a097a6b’
SPSecurityContext: Request for security token failed with exception: System.ServiceModel.ServerTooBusyException: The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc is unavailable.  This could be because the service is too busy or because no endpoint was found listening at the specified address. Please ensure that the address is correct and try accessing the service again later. —> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.     at System.Net.HttpWebRequest.GetResponse()     at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)     — End of inner exception stack trace —    Server stack trace:      at System.ServiceModel.Chann…
…els.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)     at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)     at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown at [0]:      at System.Runtime.Remoting….
…Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse& rstr)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)     at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo, SPRequestSecurityTokenProperties properties)
An exception occurred when trying to issue security token: The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc is unavailable.  This could be because the service is too busy or because no endpoint was found listening at the specified address. Please ensure that the address is correct and try accessing the service again later..
Microsoft.SharePoint.PowerShell.SPCmdletPipeBindException: Cannot find an SPSite object with Id or Url: http://YourSharePoint. —> System.ServiceModel.ServerTooBusyException: The HTTP service located at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc is unavailable.  This could be because the service is too busy or because no endpoint was found listening at the specified address. Please ensure that the address is correct and try accessing the service again later. —> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.     at System.Net.HttpWebRequest.GetResponse()     at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)     — End of inner exception stack…
… trace —    Server stack trace:      at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)     at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)     at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage mess…
…age)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse& rstr)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)     at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo, SPRequestSecurityTokenProperties properties)     at Microsoft.SharePoi…
…nt.SPSecurityContext.SecurityTokenForLegacyLoginContext(Uri context)     at Microsoft.SharePoint.SPSite.InitUserToken(SPRequest request)     at Microsoft.SharePoint.SPSite.SPSiteConstructor(SPFarm farm, Guid applicationId, Guid contentDatabaseId, Guid siteId, Guid siteSubscriptionId, SPUrlZone zone, Uri requestUri, String serverRelativeUrl, Boolean hostHeaderIsSiteName, SPUserToken userToken, Boolean appWebRequest, String appHostHeaderRedirectDomain, String appSiteDomainPrefix, String subscriptionName, String appSiteDomainId, Uri primaryUri)     at Microsoft.SharePoint.SPSite..ctor(SPFarm farm, Uri requestUri, Boolean contextSite, Boolean swapSchemeForPathBasedSites, SPUserToken userToken)     at Microsoft.SharePoint.SPSite..ctor(SPFarm farm, Uri requestUri, Boolean contextSite, SPUserToke…
…n userToken)     at Microsoft.SharePoint.SPSite..ctor(String requestUrl)     at Microsoft.SharePoint.PowerShell.SPSitePipeBind.Read(Boolean exactUrl)     — End of inner exception stack trace —     at Microsoft.SharePoint.PowerShell.SPSitePipeBind.Read(Boolean exactUrl)     at Microsoft.SharePoint.PowerShell.SPCmdletGetSite.InternalValidate()     at Microsoft.SharePoint.PowerShell.SPCmdlet.ProcessRecord()
Error Category: InvalidData    Target Object  Microsoft.SharePoint.PowerShell.SPCmdletGetSite  Details  NULL  RecommendedAction NULL
Leaving ProcessRecord Method of Get-SPSite.
Entering EndProcessing Method of Get-SPSite.
Leaving EndProcessing Method of Get-SPSite.

Checking the web server I found that the application pool “SecurityTokenServiceApplicationPool” was accidentally stopped, causing issues with the authentication. Restarting it solved my problem.

After solving the issue I played further with the token service, and stopped the web site for "SharePoint Web Services" in IIS, just to learn how the error messages differ in this case. Here is what I found.

The message for the failure of the the Get-SPSite / Get-SPWeb cmdlets is the same as the original case, however the stack trace ($StackTrace) is an other one:

   at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecut
eEnumerate(Object input, Hashtable errorResults, Boolean enumerate)
   at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boo
lean ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] p
ipeElementAsts, CommandRedirection[][] commandRedirections, FunctionContext fun
cContext)
   at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(Inte
rpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.
Run(InterpretedFrame frame)

Trying to enumerate the site collection in the web application (see above) results in this error:

An error occurred while enumerating through a collection: There was no
endpoint listening at
http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc that
could accept the message. This is often caused by an incorrect address or SOAP
action. See InnerException, if present, for more details..
At line:1 char:1
+ $wa.Sites
+ ~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Microsoft.Share…rePoint.SPS
   ite]:SPEnumerator`1) [], RuntimeException
    + FullyQualifiedErrorId : BadEnumeration

And finally the ULS trace for this scenario (as you can see this time for Get-SPWeb instead of Get-SPSite) :

Entering BeginProcessing Method of Get-SPWeb.
Leaving BeginProcessing Method of Get-SPWeb.
Entering ProcessRecord Method of Get-SPWeb.
SecurityTokenServiceSendRequest: RemoteAddress: ‘http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc&#8217; Channel: ‘Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustChannelContract’ Action: ‘http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue&#8217; MessageId: ‘urn:uuid:bc713eb8-c1f1-453c-b333-86ef8e4411a5’
SPSecurityContext: Request for security token failed with exception: System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. —> System.Net.WebException: Unable to connect to the remote server —> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:32843     at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)     at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address,…
… ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)     — End of inner exception stack trace —     at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)     at System.Net.HttpWebRequest.GetRequestStream()     at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()     — End of inner exception stack trace —    Server stack trace:      at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()     at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)     at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)     at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) …
…    at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, Re…
…questSecurityTokenResponse& rstr)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)     at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo, SPRequestSecurityTokenProperties properties)
An exception occurred when trying to issue security token: There was no endpoint listening at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details..
Microsoft.SharePoint.PowerShell.SPCmdletPipeBindException: Cannot find an SPSite object that contains the following Id or Url: http://YourSharePoint. —> System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. —> System.Net.WebException: Unable to connect to the remote server —> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:32843     at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)     at System.Net.ServicePoint.Conne…
…ctSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)     — End of inner exception stack trace —     at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)     at System.Net.HttpWebRequest.GetRequestStream()     at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()     — End of inner exception stack trace —    Server stack trace:      at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()     at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)     at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeo…
…ut)     at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.IdentityModel.Protocols.WSTrust.IWSTrustContract.Issue(Message message) …
…    at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse& rstr)     at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel.Issue(RequestSecurityToken rst)     at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForContext(Uri context, Boolean bearerToken, SecurityToken onBehalfOf, SecurityToken actAs, SecurityToken delegateTo, SPRequestSecurityTokenProperties properties)     at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForLegacyLoginContext(Uri context)     at Microsoft.SharePoint.SPSite.InitUserToken(SPRequest request)     at Microsoft.SharePoint.SPSite.SPSiteConstructor(SPFarm farm, Guid applicationId, Guid contentDatabaseId, Guid siteId, Guid siteSubscriptionId, SPUrlZone zone, Uri requestUri, String s…
…erverRelativeUrl, Boolean hostHeaderIsSiteName, SPUserToken userToken, Boolean appWebRequest, String appHostHeaderRedirectDomain, String appSiteDomainPrefix, String subscriptionName, String appSiteDomainId, Uri primaryUri)     at Microsoft.SharePoint.SPSite..ctor(SPFarm farm, Uri requestUri, Boolean contextSite, Boolean swapSchemeForPathBasedSites, SPUserToken userToken)     at Microsoft.SharePoint.SPSite..ctor(SPFarm farm, Uri requestUri, Boolean contextSite, SPUserToken userToken)     at Microsoft.SharePoint.SPSite..ctor(String requestUrl)     at Microsoft.SharePoint.PowerShell.SPSitePipeBind.Read(Boolean exactUrl)     — End of inner exception stack trace —     at Microsoft.SharePoint.PowerShell.SPSitePipeBind.Read(Boolean exactUrl)     at Microsoft.SharePoint.PowerShell.SPCmdletGetW…
…eb.InternalValidate()     at Microsoft.SharePoint.PowerShell.SPCmdlet.ProcessRecord()
Error Category: InvalidData    Target Object  Microsoft.SharePoint.PowerShell.SPCmdletGetWeb  Details  NULL  RecommendedAction NULL
Leaving ProcessRecord Method of Get-SPWeb.
Entering EndProcessing Method of Get-SPWeb.
Leaving EndProcessing Method of Get-SPWeb.

Hopefully these details help someone to solve the issue easier and faster when it comes.

Re-creating the Missing Out-of-the-Box Search Locations using PowerShell

Filed under: PowerShell, Reflection, Search, SP 2013 — Tags: , , , — Peter Holpar @ 22:40

Recently we migrated a SharePoint 2010 web application into an existing SharePoint 2013 farm to free up the old SharePoint 2010 farm for “recycling” (it was the last application running in the old farm). The web application has a single site collection, and contains a single business application with views and list forms having a lot of customizations via “jQuery magic”. Since a replacement of the business application is planed for the near future, we decided not to upgrade the site collection to the SharePoint 2013 user interface (version 15). Leaving it in the SharePoint 2010 mode (version 14) ensures the views and forms are working further without any modifications in the JavaScript codes. After a few days a user complained, that when searching the local web site an error is displayed instead of the search results on the _layouts/OSSSearchResults.aspx page:

Unable to display this Web Part. To troubleshoot the problem, open this Web page in a Microsoft SharePoint Foundation-compatible HTML editor such as Microsoft SharePoint Designer. If the problem persists, contact your Web server administrator

In the ULS logs we found these entries:

CoreResultsWebpart: Couldnt find location with internal name LocalSearchIndex
CoreResultsDatasourceView: Couldnt find location with internal name LocalSearchIndex

On the web we found a post from Sushant Dukhande with the description of the issue, and a suggestion for the solution.

Using the script on that site it turned out, that the LocationConfigurations property of the search proxy is really empty. In an other environment, where we tested the migration we had no such issue.

Sushant Dukhande suggests to re-provision the search application. It might really solve the problem, however in our case I felt it to be an intense change, and searched for an alternative solution. Having a look into what happens under the cover of a provisioning process, I found the method responsible for provisioning the missing search locations. It is the internal static CreateOOBLocations method of the Microsoft.Office.Server.Search.Administration.LocationFactory class (in the Microsoft.Office.Server.Search assembly).

First, we need a reference to the search service application. You can get it like this (assuming it is named "Search Service Application"):

$ssa = Get-SPEnterpriseSearchServiceApplication "Search Service Application"

or via this script (as long as you are sure, you have a single instance of this service application type in your farm):

[Microsoft.Office.Server.Search.Administration.SearchServiceApplication]$ssa = Get-SPServiceApplication | ? { $_.TypeName -eq "Search Service Application" }

To display the names of the existing search locations:

$locConfigs = $ssa.LocationConfigurations
$locConfigs | % { $_.InternalName }

The following PowerShell script shows how to invoke the CreateOOBLocations method passing the search service application as parameter using PowerShell and Reflection:

$searchAssembly = [Microsoft.Office.Server.Search.Administration.SearchServiceApplication].Assembly
$locationFactory_Type = $searchAssembly.GetType("Microsoft.Office.Server.Search.Administration.LocationFactory")

$bindingFlags = [Reflection.BindingFlags]::NonPublic -bor [Reflection.BindingFlags]::Static
$mi_CreateOOBLocations = $locationFactory_Type.GetMethod("CreateOOBLocations", $bindingFlags)
$mi_CreateOOBLocations.Invoke($null, @([Microsoft.Office.Server.Search.Administration.SearchServiceApplication]$ssa))

Invoking the CreateOOBLocations method might be not always the solution for you. The same is true for the re-provisioning process suggested by Sushant Dukhande, since it invokes the same method as well. The problem, that this method has a condition, before provisioning all of the default search locations:

if (searchApp.LocationConfigurations.Count < 1)
{
    LocationConfigurationCollection locationConfigurationsInternal = searchApp.GetLocationConfigurationsInternal(true);
    CreateLiveLocation(locationConfigurationsInternal);
    CreateLiveSuggestionsLocation(locationConfigurationsInternal);
    CreateLocalSharepointLocation(locationConfigurationsInternal);
    CreateLocalPeopleLocation(locationConfigurationsInternal);
    CreateLocalFSSharePointLocation(locationConfigurationsInternal);
}

I don’t see, how our farm “lost” its search locations, but if it is possible to “lose” only a subset of the search locations (for example, only the one called LocalSearchIndex), it won’t be re-created by the CreateOOBLocations method, as the count of search location is still not zero.

In this case, the solution may be to re-create only the missing search location via the corresponding method. In the case of the LocalSearchIndex search location it is the CreateLocalSharepointLocation method of the LocationFactory class:

$locConfigs = $ssa.LocationConfigurations
$mi_CreateLocalSharepointLocation = $locationFactory_Type.GetMethod("CreateLocalSharepointLocation", $bindingFlags)
$mi_CreateLocalSharepointLocation.Invoke($null, @([Microsoft.Office.Server.Search.Administration.LocationConfigurationCollection]$locConfigs))

After fixing the issue in the farm, I’ve tested our other farms as well to find out, whether they are affected by the same problem or not. In one of the farm, the script provided in the post I mentioned earlier detected the issue, although I was sure, there is no problem with the search. It turned out to be a false positive test. This farm has its search service as a shared service from another farm, and the user account the script was run with had no permission on the search service in that remote farm. The script simply hid away the access denied error.

However, if we create a LocationConfigurationCollection instance via its internal constructor (either with a parameter of type SearchServiceApplication or of type SearchServiceApplicationProxy), the access denied error is displayed in the case the user has no permissions, and the items of the collection can be accessed if there is no problem with the permissions.

Let’s see first the script using the SearchServiceApplication:

$bindingFlags = [Reflection.BindingFlags]::NonPublic -bor [Reflection.BindingFlags]::Instance
$ci_LocationConfigurationCollection = [Microsoft.Office.Server.Search.Administration.LocationConfigurationCollection].GetConstructor($bindingFlags, $null, @([Microsoft.Office.Server.Search.Administration.SearchServiceApplication]), $null)
$locConfigs = $ci_LocationConfigurationCollection.Invoke(@([Microsoft.Office.Server.Search.Administration.SearchServiceApplication]$ssa))
$locConfigs | % { $_.InternalName }

As I wrote, you can achieve the same via a service proxy. It is useful for example, if the application is connected to a shared search service of another farm. First, we get the proxy as:

$url = "http://YourSharePointApp/&quot;
$site = Get-SPSite $url
$serviceContext = [Microsoft.SharePoint.SPServiceContext]::GetContext($site)
$ssaAppProxy = $serviceContext.GetDefaultProxy([Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy])

Next, we can use the same script as earlier, but in this case we invoke the internal constructor having the SearchServiceApplicationProxy parameter type:

$bindingFlags = [Reflection.BindingFlags]::NonPublic -bor [Reflection.BindingFlags]::Instance
$ci2_LocationConfigurationCollection = [Microsoft.Office.Server.Search.Administration.LocationConfigurationCollection].GetConstructor($bindingFlags, $null, @([Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy]), $null)
$locConfigs = $ci2_LocationConfigurationCollection.Invoke(@([Microsoft.Office.Server.Search.Administration.SearchServiceApplicationProxy]$ssaAppProxy))
$locConfigs | % { $_.InternalName }

March 15, 2016

How to get the value of a Project Server Enterprise Custom Field via the Project Server Managed Client Object Model

Filed under: Managed Client OM, Project Server — Tags: , — Peter Holpar @ 22:20

About two years ago I posted a code about how to set the value of a Project Server Enterprise Field via the managed client OM. Again and again I get the question how to get the value, once it is set already.

In the first case I assume, you already know the ID of your project and the internal name of the field you would like to query. In this case, you need only send a single request to the server, as shown in this code:

  1. var url = @"http://YourProjectServer/pwa&quot;;
  2. var projectContext = new ProjectContext(url);
  3.  
  4. var projId = new Guid("98138ffd-d0fa-e311-83c6-005056b45654");
  5. var cfInternalName = "Custom_b278fdf35d16e4119568005056b45654";
  6.  
  7. var proj = projectContext.Projects.GetByGuid(projId);
  8. projectContext.Load(proj, p => p[cfInternalName], p => p.Name);
  9.  
  10. projectContext.ExecuteQuery();
  11.  
  12. Console.WriteLine(proj.Name, proj.FieldValues[cfInternalName]);

If either the ID of your project or the internal name of the field is unknown, you need an extra round-trip before the query shown in the previous code to determine their value. In the code below I assume you know none of these values:

  1. var url = @"http://YourProjectServer/pwa&quot;;
  2. var projectContext = new ProjectContext(url);
  3. var projName = "Your Project Name";
  4. var fieldName = "NameOfTheField";
  5.  
  6. projectContext.Load(projectContext.Projects, ps => ps.Include(p => p.Id, p => p.Name));
  7. projectContext.Load(projectContext.CustomFields, cfs => cfs.Include(cf => cf.InternalName, cf => cf.Name));
  8. projectContext.ExecuteQuery();
  9.  
  10. var projId = projectContext.Projects.First(p => p.Name == projName).Id;
  11. var cfInternalName = projectContext.CustomFields.First(cf => cf.Name == fieldName).InternalName;
  12.  
  13. var proj = projectContext.Projects.GetByGuid(projId);
  14. projectContext.Load(proj, p => p[cfInternalName], p => p.Name);
  15.  
  16. projectContext.ExecuteQuery();
  17.  
  18. Console.WriteLine(proj.Name, proj.FieldValues[cfInternalName]);

I hope it helps to read the custom field values, for example the values set by the code in the former post.

Older Posts »

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 62 other followers