Putting Metadata to Work: Developing with Metadata Using the SharePoint CSOM

SharePoint has always had fantastic built-in support for metadata, but many organizations have not yet harnessed the power of metadata to build new efficiency, productivity and security into their business.  This post is the fourth in a series that will take us on a tour of what metadata is, how organizations can take advantage of its benefits and the SharePoint 2013 features that support it.

Developing with Metadata Using the SharePoint CSOM
Last year I was assisting a friend with some SharePoint development.  It involved creating and managing SharePoint metadata columns programmatically.  As well, it involved populating data within those fields when files were uploaded to SharePoint outside of the regular SharePoint upload mechanisms.  I found at the time that some of the development approaches I had to take weren't exactly obvious or what you'd expect. 
 
So, within this article I'll discuss the basics of programmatically managing metadata columns within a SharePoint list or library using the SharePoint 2013 Client Side Object model (CSOM).
 
Basic Client Object Model Concepts
The SharePoint client object model allows you to build applications for SharePoint that access and interact with SharePoint data without having to install any code on the SharePoint server.  A client object model application is designed to run on a client computer, and not on the SharePoint server itself.  I have chosen to write my client object model code on the Microsoft .NET framework using C#, but you could also create a client object model application in Silverlight or JavaScript. 
 
The client object model provides classes and APIs that can access information about site collections, sites, lists/libraries and items.  An application will call the APIs to perform one of the following operations:

• Call methods and get return values
• Call methods with a CAML query (Collaborative Application Markup Language) and get the query results
• Get and set properties

Once client object model APIs have been called to perform a specific task SharePoint will bundle up the calls into XML and send a request to the SharePoint server.  The SharePoint server receives the request, makes the appropriate calls to the SharePoint Server Object Model, bundles the results up into a JSON object and sends that object back to the client object model on the client computer.  The client object model then parses the returned JSON and provides the results back to the application as .NET Framework objects. 
 
The client object model will bundle multiple method calls into a single request to the server in order to optimize performance and reduce network traffic.  Its important to note that as the developer you control when the SharePoint client object model sends XML requests to the server, and when it receives JSON back from the server.
 
Since a client object model application will run on a client computer you do need to ensure that the user running the client object application has appropriate permissions to the data that the client object model application will access.  Otherwise you will receive access denied exceptions.
 
Typically a class that is going to access the CSOM may look a little like this:

using System;
using Microsoft.SharePoint.Client;

class CSOM_Example
{
    private string _SPLocation;
    private ClientContext _context;
    private Web _rootweb;
    private Web _site;
    public CSOM_Example(string strURL, string strSite, string strList)
    {
        _SPLocation = strURL;
        ClientContext _context = new ClientContext(_SPLocation);

        _rootweb = clientContext.Web;     //the root web = the site collection
        _context.Load(site);
        _context.ExecuteQuery();
        //Using the default authentication method means the user account that
        //the assembly is run under must have sufficient rights in SharePoint to create   
        //sites, create libraries, add files and/or get and set metadata       
       _context.AuthenticationMode = ClientAuthenticationMode.Default;

        //Check for null, and if have a rootweb then we can get the site name passed in by title
        if(_rootweb != null)
        {
                _site = GetSitebyTitle(strSite);
        }

        //check for null if the site requested does not exist
        if(_site != null)
        {
            _list = GetListByTitle(strList);
        }

       // 
        //Now we have a client context, the root web, the site and list passed in we can make
        //calls to access objects and information within the site collection, site or list
        //
        //...
        //   
    }

    //
    //Helper method: get the site object by name; there are built-in methods for this
    //but I've found this a bit more versatile
    private Web GetSiteByTitle(string strSite)
    {
        try
        {
            if (_rootweb != null)
            {
                var query = _context.LoadQuery(_rootweb.Webs.Where(p => p.Title == strSite));
                _context.ExecuteQuery();
                return query.FirstOrDefault();
            }
        }
        catch (Exception e)
        {
                //perhaps do some logging here
                return null;
        }
        return null;
    }
    //
    //Helper method: get the site object by name; there are built-in methods for this
    //but I've found this a bit more versatile    private Web GetListByTitle(string strList)
    {
        try
        {
            if (_site != null)
            {
                var query = _context.LoadQuery(_site.Lists.Where(p => p.Title == strList));
                _context.ExecuteQuery();
                return query.FirstOrDefault();
            }
        }
        catch (Exception e)
        {
                //perhaps do some logging here
                return null;
        }
        return null;
    }
    // 
    //Now we have a client context, the root web, the site and list passed in we can add
    //additional methods to access objects and information within the site collection, site or list
    //
    //...
    //
}

Client object model applications usually follow this pattern: setup your context, get your root web, make a query (like your querying a database) or set properties (like your setting database fields).

Now that we have a shell of a class that we can use to access information and objects within the site collection, site and list.  I have wrapped this code in a class with a constructor and private members that store the client context and the root web for the life of any instance of my CSOM_Example object.  You could just as easily call these APIs as shown here in a static method.

Now that we have the basic objects needed, lets see how we use them to interact with metadata.


Creating Metadata Fields
Let's start with some simple code that creates new metadata columns in an existing list.  We'll be using the _list member that we established by name in the class above.

    //First check _context and _list are not null before proceeding... leaving this code to you

    //Add metadata columns to our list - 4 different types of metadata columns
   
    //Text column type
    Field metadataField1 = _list.Fields.AddFieldAsXml(
        @"<Field Type='Text'
                 DisplayName='Address'
                 Name='Address'
                 Required='True'
                 MaxLength='256'
         </Field>",
          true,                                                    //add to default list view
          AddFieldOptions.DefaultValue);               

    //Choice column type
    Field metadataField2 = _list.Fields.AddFieldAsXml(
        @"<Field Type='Choice'
                 DisplayName='Classification'
                 Name='Classification'
                 Required='True'
                 Format='Dropdown'>
            <Default>Public</Default>
            <CHOICES>
              <CHOICE>Public</CHOICE>
              <CHOICE>Confidential</CHOICE>
              <CHOICE>Restricted</CHOICE>
            </CHOICES>
          </Field>",
          true,                                                    //add to default list view
          AddFieldOptions.DefaultValue);

    //Note column type
    Field metadataField3 = _list.Fields.AddFieldAsXml(
        @"<Field Type='Note'
                 DisplayName='Description'
                 Name='Description'
                 Required='True'
                 MaxLength='500'
                 NumLines='10'
         </Field>",
          true,                                                    //add to default list view
          AddFieldOptions.DefaultValue);

    //Number column type
    Field metadataField4 = _list.Fields.AddFieldAsXml(
        @"<Field Type='Number'
                 DisplayName='Budget'/>"
                 Name='Budget'
         </Field>",
           true,                                                    //add to default list view
           AddFieldOptions.DefaultValue);
 
    _context.ExecuteQuery();

This code is intended to be added to a new method in the class.  There are 4 different column types we're setting up here, each with their own properties specified in the XML that is passed to the method AddFieldAsXml( ).  We've created a Text field, a Choice field, a Note field (which is the multi-line text field) and a Number field.  The 2nd last parameter, which we have always set to true above is a useful one - it determines if the metadata column is displayed as part of the default view.  For more information on SharePoint 2013 column types refer to: List Item Column Type Reference.

Note: we are passing both a DisplayName and a Name attribute when creating the column.  The Name attribute is the internal name.  A SharePoint metadata column has both, and its important to know that typically when accessing column data the client object model will typically use the Internal Name.  When they're the same this isn't an issue, but read further.

Note: there are other useful properties available as well when setting up metadata columns that are not shown here.  Sometimes you may wish to create a metadata column which appears in a list or libraries default view and displays its currently set value, but you do not wish it to appear in the library settings column configuration panel, nor do you wish the user to interact with it when they Edit Properties on an  item.  This is often needed when column values are only set programmatically and you would like the user to be able to view them but not edit them.  This can be done by passing in the following attributes within the XML as well:

• ShowInEditForm='FALSE'
• ShowInNewForm='FALSE'
• ShowInDisplayForm='FALSE'

The XML for such a field would look something like this:

    Field metadataField4 = _list.Fields.AddFieldAsXml(
        @"<Field Type='Number'
                 DisplayName='Budget'/>"
                 Name='Budget'
                 ShowInEditForm='FALSE'
                 ShowInNewForm='FALSE'
                 ShowInDisplayForm='FALSE'
           </Field>",
           true,                                                    //add to default list view
           AddFieldOptions.DefaultValue);


Getting List Item Metadata Fields
Next let's look at how we programmatically get metadata values for a specific list item from existing metadata columns.

    //First check _context and _list are not null before proceeding... leaving this code to you

    //get the list item for which we want to set metadata using a CAML query
    CamlQuery camlQuery = new CamlQuery();
    camlQuery.ViewXml =        @"<View>
            <Query>
              <Where>
                <Eq>
                  <FieldRef Name='Title'>
                  <Value Type='Text'>'Fiscal2014'</Value>
                </Eq>
              </Where>
            </Query>
            <RowLimit>1</RowLimit>
          </View>";

    Microsoft.SharePoint.Client.ListItemCollection listItems = _list.GetItems(camlQuery);

    //since RowLimit is set to 1 it will retrieve the first matching item; if items are not unique
    //then you can retrieve more by increasing the RowLimit value

    //if you are working with a large list make more use of the RowLimit element - you should
    //not attempt to retrieve more than 2000 items at a time; if you need to retrieve more
    //items than that, then implement a paging algorithm

    //we can specify the metadata columns to retrieve as part of the query in Load 
    _context.Load(listItems, items => items.Include(
                 item => item["Address"],
                 item => item["Classification"],
                 item => item["Description"],
                 item => item["Budget"]));

    _context.ExecuteQuery();

    //let's assign the values of those metadata columns to our method variables - using a for 
    //loop in case got more than 1 value returned by changing RowLimit
    string    strItemAddress;
    string    strItemClassification;
    string    strItemDescription;
    Double dBudget;

    foreach (ListItem listItem in listItems)
    {
        strItemAddress = listItem["Address"]);
        strItemClassification = listItem["Classification"]);
        strItemDescription = listItem["Description"]);
        dBudget= listItem["Budget"]);      //Budget is a Number column - returns null or a double
    }

Again, this code is pretty straight forward.  We setup our CAML query looking for an item with a particular Title, we load the query specifying at the same time which metadata column data to retrieve with the item, we execute the query and then finally parse out the data.

Note: We are specifying the metadata columns to access in the _clientContext.Load() method using the column's Internal Name.  When the column was created, if only a Display Name was specified then SharePoint will create the Internal Name from the Display Name.  If the Display Name contained a space, then SharePoint will replace the space with _x0020_ and we must take this into account when specifying the column name in such calls.  For example, if the column's Display Name is 'Middle Name' and we let SharePoint create the Internal Name, then we must specify the column as such:

    _context.Load(listItems, items => items.Include(item => item["Middle_x0020_Name"]));

If when creating the column we specified an Internal Name that's different from the Display Name then we must use the Internal Name.  For example, if DisplayName='Middle Name' and InternalName = 'MiddleName' then you must use:

    _context.Load(listItems, items => items.Include(item => item["MiddleName"]));
 

Setting List Item Metadata Fields
Finally, let's look at some simple code to update metadata fields for an existing item in an existing list.

    //First check _context and _list are not null before proceeding... leaving this code to you

    //get the list item for which we want to set metadata using a CAML query
    CamlQuery camlQuery = new CamlQuery();
    camlQuery.ViewXml =        @"<View>
            <Query>
              <Where>
                <Eq>
                  <FieldRef Name='Title'>
                  <Value Type='Text'>'Fiscal'</Value>
                </Eq>
              </Where>
            </Query>
            <RowLimit>100</RowLimit>
          </View>";

    Microsoft.SharePoint.Client.ListItemCollection listItems = _list.GetItems(camlQuery);

     //since RowLimit is set to 100 it will retrieve the first 100 matching items
    //you can retrieve more by increasing the RowLimit value

    //if you are working with a large list make more use of the RowLimit element - you should
    //not attempt to retrieve more than 2000 items at a time; if you need to retrieve more
    //items than that, then implement a paging algorithm    _context.Load(listItems);
    _context.ExecuteQuery();

   //execute query first in order to get item, and then execute again to  update
 
    //for each matching list item returned set metadata values to some parameters passed in
    foreach (Microsoft.SharePoint.Client.ListItem listItem in listItems)
    {
        listItem["Address"] = strMetadataValue1;
        listItem["Classification"] = strMetadataValue2;
        listItem["Description"] = strMetadataValue3;
        listItem["Budget"] = dMetadataValue4.ToString();
        listItem["Middle_x0020_Name"] = strMetadataValue5;

        //include more metadata fields here if necessary...
    }

   //execute the update once for all the list items retrieved

   listItem.Update();
    _context.ExecuteQuery();

You can see that again we follow a similar pattern, where we setup our CAML query to get the item in question.  In this case we're getting the first 100 items that match our query.  Certainly our CAML query can get more complex and specific than this.  For more information on CAML query structure refer to the Collaborative Application Markup Language Reference.  Then we execute our query to get the item, we set our metadata field values, we update the list item and then we execute our query again.

There are many business applications to working with metadata programmatically within SharePoint, especially when metadata is populated from external sources or other line of business systems, or when metadata must be extracted from SharePoint and used in other external systems.  Hopefully these code examples can help you get started with integrating SharePoint metadata into your business processes.

     -Antonio