Blog Medium Thumbnails

Manipulating SharePoint External Lists using CSOM

This is the second part in my series on developing code to modify external lists.  In this post we will be updating SharePoint external lists using CSOM.  Like standard PowerShell there is not any real difference in the execution of the update between internal and external lists.  What is different is how the data is obtained when performing an update, read or deletion.  Like PowerShell, in CSOM you can find the item you wish to access using the ID.  However, because there isn’t a list ID field in an external list you have to query to find the item(s) you need.  You could loop through each item in the list until you find one with the particular attributes you are looking for, but that is not efficient and it may be difficult if the fields you are checking have multiple rows with the same values.  So brush up on your CAML query.

Using CSOM to Manage Data in Internal Lists

CSOM has come a long way since its inception in SharePoint 2010.  I would still be recommending it over REST except that unlike REST things are a bit different between on-prem and SharePoint Online CSOM versions (you need a different versions for each).  This post isn’t meant as a discussion on the pros and cons of REST vs CSOM though (You can get more information here if you are curious though).  CSOM is probably a little more complicated than the PowerShell examples I provided in my previous post, but one you get the hang of loading the data you need before working with it the process is pretty easy.

To show you what we are going to be doing here’s the list we will be starting off with:

Updating SharePoint External Lists using CSOM - Internal List Before Code

Using this config:

we will update the list with the following code:

Running the code:

Updating SharePoint External Lists using CSOM - Internal List Running Code

The list gets updated as expected:

Updating SharePoint External Lists using CSOM - Internal List After Code


Using CSOM to Update External Lists

As I stated above, the code is very similar to internal lists except there you need to use CAML Query to access the items.  Note with CSOM you don’t have to define the ServiceContext scope.

The list:

Updating SharePoint External Lists using CSOM - External List Before Code

Will get modified based on this config:

Reviewing the code, note the insertion of the CAML Query to find a specific item.  One important thing to note with the CAML query.  You have to use the name of the function used to define the Read-List process in the External Content Type

Running the code:

Updating SharePoint External Lists using CSOM - External List Running Code

Leaves us with the following updated list:

Updating SharePoint External Lists using CSOM - External List After Code

So again, other than gathering the item to update, managing items in an external list is very similar to managing items in an internal list.  My next post we will cover the differences in manipulating internal and external items with REST.

Thanks for reading!!

A Little Bit About Me

David humbly considers himself a SharePoint JoAT (Jack of All Trades). He is the SharePoint Services Lead at Solvera Solutions, and an Office Servers and Services MVP. Over the last 16 years, David has worked in a wide variety of areas in IT ranging from DesktopServer support to developing in C++ and .NET technologies. Over that period, David has been able to add value to a wide variety of clients in several sectors, including utilities, government, banking, and agriculture.

Since early 2010, David has been working exclusively in the SharePoint domain. He has helped stand up multiple Enterprise SharePoint environments as well as designed and built numerous SharePoint solutions. David’s extensive experience in SharePoint is complemented by a range of SharePoint training courses and Certification in Microsoft SharePoint 2010. He is a knowledgeable and sought-after international speaker within IT circles and consulted regularly by companies formulating their SharePoint strategies.

Using PowerShell to Access Data In External Lists

I created my  blog series on SharePoint BCS quite some time ago in order to illustrate how easy it is to access data outside of SharePoint and present it within a list or document library.  I have been meaning for quite some time to demonstrate how writing code for external lists\libraries is very similar to writing code for internal lists\libraries.  Today I am going to start a new three part series on SharePoint BCS development by showing you how you can enhance your solutions using PowerShell to access data in external lists.  The posts aren’t going to be too in-depth, but meant to get you started.  I hope to build on them later and post a series on creating an entire solution around BCS.

Using PowerShell to Manage Data in Internal Lists

With SharePoint PowerShell you are basically using the server object model for development.  So this means you have access to the methods and processes utilised by this system.  What you’ll notice is that code to perform inserts, updates, reads and deletions is pretty much exactly the same.  Where the code differs between internal and external is actually around accessing the item to be updated, read or deleted.

To illustrate I have created a script that performs an insert, update, read and delete.  To show you what’s happening here’s the config for the code:


As you can see the code should insert a new item, update two, read a third and finally delete.  The code is pretty straight forward:

Using the following list:

Using PowerShell to Access Data In External Lists - Internal List Before Code

And running the above script:

Using PowerShell to Access Data In External Lists - InternalListRunningCode

We end up with the result:

Using PowerShell to Access Data In External Lists - Internal List After Code


Using PowerShell to Update External Lists

The code is almost identical.  What actually changes is the code used for finding the items we want to update.  We also have to define the ServiceContext scope to process the transactions.  External list manipulation requires the scope to import the data catalogue (more information can be found here).  First the config to see what the plan is:

Then using the code:

Notice in the code above that we are using a pipe to gather the data.  One thing to note with External Lists is they do not have a numerical ID field.  However, in this case we are grabbing the ID field of the Asset so we can search on the AssetID of the item.

We then update the following list:

Using PowerShell to Access Data In External Lists - External List Before Code

by running the code:

Using PowerShell to Access Data In External Lists - External List Running Code

And receive the following updated list:

Using PowerShell to Access Data In External Lists - External List After Code


That’s the basics of updating an External List with PowerShell.  In the next post I will cover updating using CSOM.


Thanks for Reading!

Introducing Microsoft365

Remember last year Microsoft announced Secure Productive Enterprise?  Well, today at Microsoft Inspire (the rebranded Partner Conference) they rebranded it and I personally think it’s a great idea.  Welcome to Microsoft365.   In a nutshell, Microsoft365 is Office 365 and Windows 10 bundled together with Mobility for the Enterprise and all wrapped up in enterprise level security.  The following is my take on how the new product focus is going to help your business moving forward.


Encouraging four main pillars the product is focused on, Satya Nadella made the announcement this morning (July 10, 2017).  Microsoft wants the tool to focus on Creativity, Teamwork, Simplicity, and Security.


Creativity of course because with the product you will get all the tools you need to create great things.  You will get the same tools you have been using already like Word, Excel, PowerPoint, OneNote, etc).  With these, you can create everything you need in your day to day work.  From proposals to meeting notes to financial status with Microsoft365, you can create all this and so much more.


With Office365 you will be getting great tools to encourage and support teamwork.  SharePoint of course but now you can increase collaboration within your teams using other fantastic tools like Microsoft Groups, Microsoft Team, OneDrive for Business, Outlook and Exchange, Skype for Business, Yammer, and so much more.  Being able to accomplish tasks as a group has never been easier.


It’s all within one package for you.  No need to buy this tool for that job, or that tool for another job.  Everything you need is in Microsoft365.  And even if it isn’t (yet ;-D ) Microsoft has made connectors to many items for you to do just that.  Microsoft teams have connectors to social media like Twitter and Facebook.  Flow can also access these systems as well.  While it may seem less than simple and when first getting started it may very well be, but as you carve out your usage of Microsoft365 you will find having everything at your fingertips in one product couldn’t make it easier for you to get things done.


With Microsoft365 you will be getting enterprise level security.  You will have Encryption, Data Loss Protection (DLP) and Information Rights Management all through Office365.  Thus fully securing your data from accidental loss, or being viewed by those who shouldn’t.  You will get great protection in your systems with Windows Information Protection, Credential Guard, App Locker and much more.  Remember that Office365 has released a fantastic security console that allows you to fully protect your environment even better than before.

What I think is fantastic is that you don’t have to purchase the items seperately.  As a business you now can group your most used individual products (Windows 10, business applications (Word, Excel, etc), collaboration tools like SharePoint, OneDrive, Skype, etc) and more all in one bundle.  It’s a great option for many businesses to use.  While not a huge new product for Microsoft, I think that the rebranding and launch of Microsoft365 is absolutely fantastic and looking forward to working with businesses to build out and support solutions utilizing this product.


Thanks for reading!!

SharePoint Custom Solution Crashes IIS Worker Process (w3wp.exe) – Part 2

In Part 1 of this series, I discussed an issue we were having in one of our SharePoint 2013 farms and how I determined the issue was occurring because of a set of event receivers acting on the library.  In this post, I will discuss the code being used and what the final result was determined to be.  Stick around, it’s not what you think.

To be terribly honest nothing jumped out at me while looking over the code.  The initial review of the code indicated the issue could be around where the event receiver was trying to determine if the user adding the file was a member of the site owner group.  The original code was:

This seems pretty straight forward, but when “CheckIfUserInSPGroup” is called things aren’t quite as kosher.

Again, normally not a huge issue, except that best practices state that you shouldn’t instantiate SPSite, SPWeb or SPList objects within an Event Receiver.  The reason for this is it causes extra database calls (more information here:  I thought this could be the culprit, but wasn’t convinced.  If this was the issue, why does it work fine for years and then suddenly stop working?  The reason the code is instantiating the SPSite and SPWeb object is it is used elsewhere in the solution and could be called by users who do not have the required access.  The same goes for the event receiver.  If I do not have access to control security groups in the site, I get an UnauthorizedAccessException.

So I thought, why not just use that.  We can safely assume that if the UnauthorizedAccessException error is thrown, the user is not in the Owners group.  So I updated the code with a try\catch (why one wasn’t already being used I don’t know) and added some logic into the catch.  Not generally the best method, but when used for targeted exceptions I believe acceptable IMHO.

I also created a new method for the event receiver to call.  I couldn’t modify the existing one, as it contained valid logic to handle users without access and was being used elsewhere.

So I moved the code into Pre-Prod and tried it out.  No change.  Still hanging, throwing errors and crashing the app pool.

Next step was to install Visual Studio into Pre-Prod and attach to the IIS Worker process.  I followed the code until it got into the newly created CheckIfUserInSPGroupEvntRcvr method.  There it stayed.  It kept looping through the AD users and groups within the SharePoint group.  As it was looping I watched the worker process memory usage grow and grow until it finally crashed again.  This didn’t make any sense as there are NOT that many users in these groups.

The Cause of it All

I took a look at the ownership group for the site I was testing with.  Like most (not all) of our project sites, it contained an AD group that contains our project team.  Let’s call that group All-Project.  All-Project had about a dozen users within it, however, there was an anomaly.  It also contained the Owners group from another project site.  This was an oddity.  I took a look at the Owner group and it also contained the same All-Project group. There was the culprit.

As you can see in the code above, it is designed for nested groups, so if the code hits a group it digs down to see if the nested group contains the user.  Because this Owner group was added (in error I found out while trying to figure out why it was there) to the All-Projects group, the code would dig into All-Projects then to the Owners group, from there back into the All-Projects group and then back into the Owners group… see where I am going with this?  By adding that single group to the All-Projects group in error an infinite recursion loop was created in the code.

The Final Fix

So the final fix was not an environmental change or a code modification.  It was simply to remove the Owners group from the All-Projects group.  Once that was done, the original code functioned as designed.  If this becomes a regular occurrence I will have to update the code to handle such an event, but in this case, I didn’t.  The farm is in containment (no further development short of break\fix) and the issue has not occurred for two years before this.  I hope the steps I documented in this blog series helps others out.


Thanks for reading!