Three Columns Blog

Manipulating SharePoint External Lists using REST

In this third post on accessing SharePoint External Lists we are going to be manipulating sharePoint external lists using REST APIs.  You can view the previous two posts of this series by clicking on one of the following links:

Of all the methods of coding against external lists I would have to say that REST is probably the easiest.  You don’t have to create any complicated CAML queries to find items to read, delete or update as you would with CSOM.  You don’t have to add Service Context scope as you do with PowerShell.  In fact, with REST you can actually access data using the ID of the list item.

But Dave, I thought you said external lists don’t have IDs

Actually, they do.  However, as I said in the first post, the do not have Numerical IDs.  The IDs are actual text based.  Once fantastic about SharePoint REST API is you actually have a method you can call named: GetItemByStringId.  This method is exactly as it sounds.  Using it you will be able to get the internal ID of the list item generated when the item is added into the external list.

Unlike my previous posts I am not going to show you how to access internal lists and then compare the external access.  There isn’t a lot of code to write for this as I just want to give you the basics and Fiddler allows for that perfectly.  There is a great post on how to work with internal lists on the Office Dev Center.  You can access that here.

Manipulating SharePoint External Lists using REST

I decided that because I had done my previous posts using SharePoint on-prem that this time to show it doesn’t matter which environment you develop against, I am actually updating lists in my SharePoint Online tenant.

Just a quick disclaimer: You will notice that all of my Fiddler screenshots contain a cookie entry in the Request Header.  This is simply for accessing SharePoint Online via Fiddler.  It is not required for SharePoint on-premises.

For the purpose of this post I will be working with the following list:

Manipulating SharePoint External Lists using REST - External List

Note I have included the BDC Identity column.  For external lists, this is the item id.


Read Items

To read an item from the list we will use the endpoint: “_api/Web/Lists/getbytitle(‘<ListName>’)/GetItemByStringId(‘<item id>’)”  Special note: when using the BDC Identity, ensure it starts with a double underscore.

Manipulating SharePoint External Lists using REST - Read List Item

Manipulating SharePoint External Lists using REST - Read Item Response

Insert New Items

To insert data into an external list is actually exactly the same as it is for an internal list.  The endpoint: “_api/web/lists/getbytitle(‘SmallAsset’)/Items” is used here.  Having data in the Request Body (seen in the picture below) is what tells SharePoint to insert the data.

Manipulating SharePoint External Lists using REST - External List Insert

You can see that line 10 was added to the list:

Manipulating SharePoint External Lists using REST - External List After Insert

Update List Item

As discussed above, getting an item from an external list is a bit different.  With external lists you have to use GetByStringID instead of simply placing the item id after the indicating you are accessing items from a list in the REST call.  The call to the endpoint is similar to: “_api/Web/Lists/getbytitle(‘<List Title>’)/GetItemByStringId(‘<BDC Identity>’)“.  In the following I want to update AssetID#3 and set the location to Regina and the Details to “Updated with Rest”.

Manipulating SharePoint External Lists using REST - External List Item Update

You can see that the list item updated successfully.

Manipulating SharePoint External Lists using REST - External List After Update

Delete List Item

Deleting a list item is very similar to updating an existing item.  You simply need to change the http method to “DELETE” and remove the request body.  The call to the endpoint will be the same as the Update one in fact (“_api/Web/Lists/getbytitle(‘<List Title>’)/GetItemByStringId(‘<BDC Identity>’)“).  You can see in following screenshots that I deleted the entry that indicated it was added by Fiddler (AssetID: #9).

Manipulating SharePoint External Lists using REST - External List Delete Item

You can see the item is now removed

Manipulating SharePoint External Lists using REST - External List After Delete

That’s the basics of working with External Lists and REST APIs.  As you can see it’s really easy.  Each of the methods I illustrated today could be moved to on-prem SharePoint simply by changing the URL and nothing else.


Thanks for reading!!

Manipulating SharePoint External Lists using CSOM

This is the second part in my series on developing code to modify external lists.  In this post we will be updating SharePoint external lists using CSOM.  Like standard PowerShell there is not any real difference in the execution of the update between internal and external lists.  What is different is how the data is obtained when performing an update, read or deletion.  Like PowerShell, in CSOM you can find the item you wish to access using the ID.  However, because there isn’t a list ID field in an external list you have to query to find the item(s) you need.  You could loop through each item in the list until you find one with the particular attributes you are looking for, but that is not efficient and it may be difficult if the fields you are checking have multiple rows with the same values.  So brush up on your CAML query.

Using CSOM to Manage Data in Internal Lists

CSOM has come a long way since its inception in SharePoint 2010.  I would still be recommending it over REST except that unlike REST things are a bit different between on-prem and SharePoint Online CSOM versions (you need a different versions for each).  This post isn’t meant as a discussion on the pros and cons of REST vs CSOM though (You can get more information here if you are curious though).  CSOM is probably a little more complicated than the PowerShell examples I provided in my previous post, but one you get the hang of loading the data you need before working with it the process is pretty easy.

To show you what we are going to be doing here’s the list we will be starting off with:

Updating SharePoint External Lists using CSOM - Internal List Before Code

Using this config:

we will update the list with the following code:

Running the code:

Updating SharePoint External Lists using CSOM - Internal List Running Code

The list gets updated as expected:

Updating SharePoint External Lists using CSOM - Internal List After Code


Using CSOM to Update External Lists

As I stated above, the code is very similar to internal lists except there you need to use CAML Query to access the items.  Note with CSOM you don’t have to define the ServiceContext scope.

The list:

Updating SharePoint External Lists using CSOM - External List Before Code

Will get modified based on this config:

Reviewing the code, note the insertion of the CAML Query to find a specific item.  One important thing to note with the CAML query.  You have to use the name of the function used to define the Read-List process in the External Content Type

Running the code:

Updating SharePoint External Lists using CSOM - External List Running Code

Leaves us with the following updated list:

Updating SharePoint External Lists using CSOM - External List After Code

So again, other than gathering the item to update, managing items in an external list is very similar to managing items in an internal list.  My next post we will cover the differences in manipulating internal and external items with REST.

Thanks for reading!!

A Little Bit About Me

David humbly considers himself a SharePoint JoAT (Jack of All Trades). He is the SharePoint Services Lead at Solvera Solutions, and an Office Servers and Services MVP. Over the last 16 years, David has worked in a wide variety of areas in IT ranging from DesktopServer support to developing in C++ and .NET technologies. Over that period, David has been able to add value to a wide variety of clients in several sectors, including utilities, government, banking, and agriculture.

Since early 2010, David has been working exclusively in the SharePoint domain. He has helped stand up multiple Enterprise SharePoint environments as well as designed and built numerous SharePoint solutions. David’s extensive experience in SharePoint is complemented by a range of SharePoint training courses and Certification in Microsoft SharePoint 2010. He is a knowledgeable and sought-after international speaker within IT circles and consulted regularly by companies formulating their SharePoint strategies.

Using PowerShell to Access Data In External Lists

I created my  blog series on SharePoint BCS quite some time ago in order to illustrate how easy it is to access data outside of SharePoint and present it within a list or document library.  I have been meaning for quite some time to demonstrate how writing code for external lists\libraries is very similar to writing code for internal lists\libraries.  Today I am going to start a new three part series on SharePoint BCS development by showing you how you can enhance your solutions using PowerShell to access data in external lists.  The posts aren’t going to be too in-depth, but meant to get you started.  I hope to build on them later and post a series on creating an entire solution around BCS.

Using PowerShell to Manage Data in Internal Lists

With SharePoint PowerShell you are basically using the server object model for development.  So this means you have access to the methods and processes utilised by this system.  What you’ll notice is that code to perform inserts, updates, reads and deletions is pretty much exactly the same.  Where the code differs between internal and external is actually around accessing the item to be updated, read or deleted.

To illustrate I have created a script that performs an insert, update, read and delete.  To show you what’s happening here’s the config for the code:


As you can see the code should insert a new item, update two, read a third and finally delete.  The code is pretty straight forward:

Using the following list:

Using PowerShell to Access Data In External Lists - Internal List Before Code

And running the above script:

Using PowerShell to Access Data In External Lists - InternalListRunningCode

We end up with the result:

Using PowerShell to Access Data In External Lists - Internal List After Code


Using PowerShell to Update External Lists

The code is almost identical.  What actually changes is the code used for finding the items we want to update.  We also have to define the ServiceContext scope to process the transactions.  External list manipulation requires the scope to import the data catalogue (more information can be found here).  First the config to see what the plan is:

Then using the code:

Notice in the code above that we are using a pipe to gather the data.  One thing to note with External Lists is they do not have a numerical ID field.  However, in this case we are grabbing the ID field of the Asset so we can search on the AssetID of the item.

We then update the following list:

Using PowerShell to Access Data In External Lists - External List Before Code

by running the code:

Using PowerShell to Access Data In External Lists - External List Running Code

And receive the following updated list:

Using PowerShell to Access Data In External Lists - External List After Code


That’s the basics of updating an External List with PowerShell.  In the next post I will cover updating using CSOM.


Thanks for Reading!

Introducing Microsoft365

Remember last year Microsoft announced Secure Productive Enterprise?  Well, today at Microsoft Inspire (the rebranded Partner Conference) they rebranded it and I personally think it’s a great idea.  Welcome to Microsoft365.   In a nutshell, Microsoft365 is Office 365 and Windows 10 bundled together with Mobility for the Enterprise and all wrapped up in enterprise level security.  The following is my take on how the new product focus is going to help your business moving forward.


Encouraging four main pillars the product is focused on, Satya Nadella made the announcement this morning (July 10, 2017).  Microsoft wants the tool to focus on Creativity, Teamwork, Simplicity, and Security.


Creativity of course because with the product you will get all the tools you need to create great things.  You will get the same tools you have been using already like Word, Excel, PowerPoint, OneNote, etc).  With these, you can create everything you need in your day to day work.  From proposals to meeting notes to financial status with Microsoft365, you can create all this and so much more.


With Office365 you will be getting great tools to encourage and support teamwork.  SharePoint of course but now you can increase collaboration within your teams using other fantastic tools like Microsoft Groups, Microsoft Team, OneDrive for Business, Outlook and Exchange, Skype for Business, Yammer, and so much more.  Being able to accomplish tasks as a group has never been easier.


It’s all within one package for you.  No need to buy this tool for that job, or that tool for another job.  Everything you need is in Microsoft365.  And even if it isn’t (yet ;-D ) Microsoft has made connectors to many items for you to do just that.  Microsoft teams have connectors to social media like Twitter and Facebook.  Flow can also access these systems as well.  While it may seem less than simple and when first getting started it may very well be, but as you carve out your usage of Microsoft365 you will find having everything at your fingertips in one product couldn’t make it easier for you to get things done.


With Microsoft365 you will be getting enterprise level security.  You will have Encryption, Data Loss Protection (DLP) and Information Rights Management all through Office365.  Thus fully securing your data from accidental loss, or being viewed by those who shouldn’t.  You will get great protection in your systems with Windows Information Protection, Credential Guard, App Locker and much more.  Remember that Office365 has released a fantastic security console that allows you to fully protect your environment even better than before.

What I think is fantastic is that you don’t have to purchase the items seperately.  As a business you now can group your most used individual products (Windows 10, business applications (Word, Excel, etc), collaboration tools like SharePoint, OneDrive, Skype, etc) and more all in one bundle.  It’s a great option for many businesses to use.  While not a huge new product for Microsoft, I think that the rebranding and launch of Microsoft365 is absolutely fantastic and looking forward to working with businesses to build out and support solutions utilizing this product.


Thanks for reading!!