The Future of SharePoint Security and Governance

May the 4th Be With You

Today is typically set aside to celebrate Star Wars movies and culture that many of us have enjoyed for years.  I'm hoping to take in one of the Star Wars movies with my family this evening, once the kids finish their homework of course.

In other important news...
Today Microsoft announced the general availability of SharePoint Server 2016!

Microsoft also announced major new directions that we're going to see for SharePoint over the coming year!  People that read my ramblings know that I focus much of my work on security, so I'd like to share some of the security related capabilities that are included in Microsoft's roadmap for SharePoint. Microsoft has also reaffirmed their commitment to security, privacy and compliance with some significant new capabilities in their roadmap.

Dynamic Conditional Access Policies

One major new feature we're going to see is dynamic conditional access policies that administrators can define, allowing them to control the content that users can access based on the user's identity, the application or device they're using and their network location.

Administrators will be able to effectively prevent users from accessing high-security files in SharePoint from a mobile device or home network which the organization doesn't control, but allow the user to access those files from a corporate laptop.

Microsoft Windows Server 2012 has had this capability for years with its Dynamic Access Control (DAC) capability where you can define policies based on attributes in a user's identity (ex. security clearance) and metadata associated with a document (ex. its classification) and have those policies automatically enforced on Windows file servers.  It typically required use of the Windows File Classification Infrastructure (FCI).  As well, some third party security tools have layered these types of security policies on top of on premise SharePoint deployments in the past.

From a security perspective, it will be fantastic to see this or a similar capability finally making its way to SharePoint.

Site Classification

In an update later this year, customers will be able to classify SharePoint sites so that security policies are scoped and enforced on all content in the site. When creating a new site, whether a team site or a publishing site, you'll be able to select the classification of the site. A site's classification is typically related to the sensitivity of the content you plan to store or present within the site, which will be displayed right below the site's name by default. This will really help users to understand when they are accessing sites with sensitive corporate data.

This feature sounds simple, but its extremely significant because it allows customers to identify where sensitive data exists in their environment. Identifying where sensitive data lives is traditionally the first battle you fight, when trying to protect your sensitive corporate data. This is a great advancement in improving the governance of our SharePoint environments.

Hybrid SharePoint Insights - Hybrid Activity Monitoring and Reporting

Between the fall of 2015 and early 2016, Microsoft released the activity monitoring and reporting features within SharePoint Online and OneDrive for Business. This is a great capability for either monitoring user activity within your tenant, or performing forensic analysis into data breaches. I wrote an article about this capability here: Securing Office 365 with Activity Monitoring.

By the end of 2016, Microsoft will release a preview of Hybrid SharePoint Insights which will aggregate data from both your on premise SharePoint 2016 environment and your SharePoint Online/OneDrive for Business tenant. This will allow you to monitor and report on user activity from both your on premise and Office 365 environments through one easy to use interface.

Bring Your Own Encryption Keys

We've have heard over the last year about how Microsoft encrypts all content stored within SharePoint Online and OneDrive for Business with a complex system that partitions data, uniquely encrypts each partition with a different key, randomly distributes and stores those encrypted partitions in Azure Storage Blobs, encrypts the keys themselves and stores those in a master key store and rotates all keys every 24 hours.  I've written about this myself here:  How Does Microsoft Protect Our Data in Office 365.  This is already happening in Office 365 and its completely transparent to customers.

What's new is that later this year customers will be able to bring their own encryption keys to further lock down their data, preventing even Microsoft technical staff running the Office 365 service from accessing your data.  These continued efforts continue to help protect our data and our privacy.

Data Loss Prevention Improvements

I recently gave a webinar on the new SharePoint 2016 data loss prevention feature which can be found here: Data Loss Prevention in SharePoint 2016. Later this year you will be able to apply data loss prevention policies down to the site level. Today you can only apply those policies at the site collection level. This will allow us to get more specific about where and which content data loss prevention policies are applied to.

External Sharing Improvements

In Office 365, you can now whitelist and blacklist specific domains for external sharing. As well, later this year we'll be able to set an expiry period for external sharing, so content is only shared externally for a specific period of time.

Other Exciting Additions - New Mobile Experience, Team Sites...

There are many other welcome new additions planned over the coming year including a new SharePoint Mobile app experience, allowing users to easy access news from across the company, the sites that people use and access most, quick links to important pages and a list of their coworkers or those they collaborate most with.  This new app apparently uses Microsoft's investments in machine learning and the Office Graph to help surface the most relevant content and people for you, and present that ahead of less relevant information.  This sounds a lot like a mobile version of Delve doesn't it?!   The new mobile app will be available towards end of June 2016 for iOS, with Android and Windows versions coming later this year.  The OneDrive mobile app will also be getting enhancements through machine learning to provide users with suggestions of useful content through both OneDrive for Business and SharePoint.

As well, team sites will get a new home page which gives users a quick look at team sites which they are part of, along with updates that have been recently made to those sites.  The idea here being that users can more quickly get to the work and sites that are more relevant to them at that moment.

There's tons of other exciting updates... within Office 365, we hear that SharePoint team sites will be coming together with Office 365 groups as well - whenever a new Office 365 group is created, a new team site will be created as well. As a result, you'll be able to share team sites within Office 365 groups with external users through the Office 365 external sharing feature. This is a nice addition, but can create some security issues as well if you don't have appropriate governance in place.

We will likely see these updates come out through Microsoft's new SharePoint 2016 Feature Packs planned over the next year.

May the 4th be with you!
   -Antonio

The Dawn of Transparency

Last week we saw Uber publicly release its first Transparency Report (https://transparencyreport.uber.com/) and they've committed to release one every 6 months.  This has actually been happening for a few years.  Google began this trend for major tech companies in 2010, followed by Twitter in 2012 and now we have a number of other companies doing the same:

• Google: https://www.google.com/transparencyreport/
• Yahoo: https://transparency.yahoo.com/
• Facebook: https://govtrequests.facebook.com/
• Twitter: https://transparency.twitter.com/
• Apple: http://www.apple.com/privacy/transparency-reports/
• Microsoft: https://www.microsoft.com/about/csr/transparencyhub/

A transparency report is a public statement issued by a company, on some sort of regular basis, that discloses aggregated data (not individual instance data) about requests for user information or content. These requests are made by governmental or regulatory bodies, as well as law enforcement agencies. Transparency reports are focused on a specific period of time and typically include how frequently these agencies request data and the types of responses provided. They also include under which authority the requests were made such as subpoena, search warrants, court order or emergencies.  Disclosing a transparency report helps the general public understand the scope and authority by which regulatory bodies are permitted to access personal information that we would typically consider private.

In the last 6 months of 2015, Uber reports that it handled 415 requests for private data from various law enforcement agencies.  It provided at least a portion of the data requested in approximately 85% of cases.  Out of those requests 368 came from state run agencies, while 47 were from federal agencies.  As a result of these requests, 408 riders and 205 drivers were impacted.  As part of its report, Uber states that it makes it a policy to protect passenger privacy and requires valid and sufficient legal process from official government agencies before disclosing any information about its customers. It typically attempts to narrow the scope of data requests, which it is successful doing in some cases.

I find the release of transparency reports significant!  It means we now have major tech companies, who request and use our personal information every day, releasing information to the public that clearly describe how they handle requests for that private data.  This helps to put pressure on those technology companies retrieving our data to securely store and protect that data, and it shows that they are making attempts to do just that.  This also allows us as consumers of online services to understand the scope of government requests and to watch the trends - to see if these requests are increasing.  Finally, it sheds a light on a practice that would otherwise be kept secret, and it encourages us to put pressure on our governments and law enforcement agencies to handle our personal data with the sensitivity and care it deserves.

Consider a very simple scenario where a law enforcement agency requests data from an online service about an illegal activity related to a person named 'John Smith'.  What if your name is also 'John Smith' and you happen to use the same service?  Your personal data may get lumped in with the data provided.  You want law enforcement to be able to do its job of course.  However, you would also like to think that the data provided is under some sort of legal retention policy so after a specific amount of time, once the legal case is closed, your data is permanently deleted and you're no longer inadvertently associated with the case.  Unfortunately, many organizations take the stance of keeping data around forever, just in case.  You would like to think that the agency is taking appropriate steps to control access to that data, and storing it securely so it cannot be inappropriately exposed while in their hands.  However, agencies may not necessarily have (or follow) policies that define how personal data should be handled and secured.  You would also like to think a law enforcement agency will not disclose your data to other government agencies, but we have no guarantee of that. 

Last week we also had Microsoft announce that they are suing the US Justice Department for its frequent use of gag orders preventing it from telling people when the government obtains a warrant to read their emails.  Microsoft states that the gag order statute in the Electronic Communications Privacy Act of 1986, as employed today by the courts, is unconstitutional.  According to Microsoft, the practice violates the Fourth Amendment right of its customers to know if the government searches or seizes their property, and it breaches the company’s First Amendment right to speak to its customers.  Although the case could be in the courts for months or years, Microsoft is trying to start a public debate about the frequent use of secrecy orders in government investigations.  Microsoft reminds us that they do not own the data within their service - that the customers own their data and Microsoft is simply the custodian of that data.  Their position here very much is in line with that statement.

My personal information in many ways is my identity and I want to make sure my government does everything it can to protect it.  I for one, as a security-minded person, applaud Uber, Google, Yahoo, Facebook, Twitter, Apple, Microsoft and others for these efforts towards transparency!  The transparency report is an excellent practice which allows us to get an initial view into how personal data is accessed by our governments, regulatory bodies and law enforcement agencies.  We can begin to debate how much personal information governments should be allowed to access and what they must do with it.  Finally we can start to work with these organizations to ensure that they put in place appropriate security policies and privacy controls to better protect our personal information and identities.

Securing Office 365: Activity Monitoring

Thanks to everyone that attended my session this weekend at SharePoint Saturday San Antonio.  Thank you also to the organizers of this great event!  I really enjoyed giving the session on Office 365 Activity Monitoring and was very happy that the audience was so engaged!  Great Questions!
My slides can be found here on SlideShare: 

Securing Office365 with Activity Monitoring - SharePoint Saturday San Antonio 2016 from AntonioMaio2

If you'd like to download the presentation please click the link just below the embedded presentation.  Those who have seen my previous posts on this blog will see that I previously posted a presentation on this topic.  Microsoft has updated the Activity Monitoring feature in the Office 365 service in the last 2 months and this presentation is updated to take those updates into account.


As mentioned, Activity Monitoring is just 1 important part of securing our enterprise content management environments, but its not a "set it and forget it" activity.  Making real use of activity monitoring to help improve the security of our systems requires the right policies and procedures in place, and it requires active management and regular review of the logs.  It also requires getting the logs into some form that is not too labor intensive to retrieve, format and review.  I typically recommend the following policies:

• Review privileged user (administrator) access quarterly
• Review user access annually

Depending on the number of users in your environment, the annual access review may or may not be very practical so you may have to find some ways to make it practical, like:

• Taking a sample of users
• Developing some automated scripts or code which extract specific anomalies in the logs, like if you've identified where sensitive content exists and looking specifically for access to those areas

There are lots of other ways to make this practical, but it will likely require some serious work to put these practices into place in your specific business environment.


There were some really good questions about how you might use PowerShell to extract specific details out of the activity logs.  I'm working on a simple script to do just that now, which I'll try to post later this week.


Enjoy.
   -Antonio

Vulnerability: SharePoint 2010 and 2013 - Security Bulletin MS16-029 IMPORTANT - Mar 2016 CU

This week Microsoft released an important security bulletin related to vulnerabilities in Microsoft SharePoint 2010 and 2013, as well as Microsoft Office versions 2007, 2010, 2013, 2013RT, 2016 and 2011 & 2016 for Mac.  Full details on the vulnerabilities can be found here: https://technet.microsoft.com/en-us/library/security/ms16-029.

The following services within the listed versions of SharePoint are specifically affected:

1. Microsoft SharePoint 2010

• Word Automation Services on Microsoft SharePoint Server 2010 Service Pack 2

2. Microsoft SharePoint 2013

• Word Automation Services on Microsoft SharePoint Server 2013 Service Pack 1

3. Microsoft Office Web Apps Server 2010 Service Pack 2

• Microsoft Office Web Apps 2010 Service Pack 2

4. Microsoft Office Web Apps Server 2013 Service Pack 1

• Microsoft Web Apps Server 2013 Service Pack 1

Background Summary (from Microsoft's Bulletin)

Full details on the vulnerabilities can be found here: https://technet.microsoft.com/en-us/library/security/ms16-029. According to the official Microsoft Bulletin the following is a summary of the vulnerability:

The most severe of the vulnerabilities could allow remote code execution if a user opens a specially crafted Microsoft Office file. An attacker who successfully exploited the vulnerabilities could run arbitrary code in the context of the current user. Customers whose accounts are configured to have fewer user rights on the system could be less impacted than those who operate with administrative user rights.

A security feature bypass vulnerability exists in Microsoft Office software due to an invalidly signed binary. An attacker who successfully exploited the vulnerability could use a similarly configured binary to host malicious code. A defender would then not be able to rely on a valid binary signature to differentiate between a known good and a malicious binary. To successfully exploit this vulnerability, an attacker would have to have write access to the target location that contains the invalidly signed binary. The attacker could then overwrite the original file with their own malicious file and wait for an application, or user, to trigger the malicious binary.

The security updates provided by Microsoft address the vulnerabilities by:

• Providing a validly signed binary
• Correcting how Office handles objects in memory

Security Resources

• VULNERABILITY DETAILS: All the information you need about this vulnerability and links to the required security patch can be found here:  https://technet.microsoft.com/en-us/library/security/ms16-029. 

• UPDATE: updates addressing these vulnerabilities are available for SharePoint 2013 through the March 2016 Cumulative Update and through Windows Update.

• WORKAROUND: There is a workaround available for the Microsoft Office Memory Corruption Vulnerability.  Details of the workaround involve disabling the OLE Package function in Outlook and available at the Microsoft link provided.  The workaround would likely only assist with protecting Microsoft Office installations on desktops and not SharePoint installations from this vulnerability.

• REPORTED EXPLOITS: According to Microsoft, at this time there are no reported exploits that have occurred using these vulnerabilities.

Additional details regarding the SharePoint related vulnerabilities are available at the National Vulnerability Database at the following links:

• Microsoft Office Memory Corruption Vulnerability - CVE-2016-0021
• Microsoft Office Memory Corruption Vulnerability - CVE-2016-0134 
• Microsoft Office Security Feature Bypass Vulnerability - CVE-2016-0057

Vulnerability: SharePoint 2007, 2010 and 2013 - Security Bulletin MS16-015 CRITICAL - Feb 2016 CU

This week Microsoft released a critical security bulletin related to vulnerabilities in several versions of Microsoft Office (2007, 2010, 2013, 2013RT, 2016, 2011 for Mac, 2016 for Mac).  In addition, SharePoint 2007, 2010 and 2013 are also affected.  Full details on the vulnerabilities can be found here: https://technet.microsoft.com/library/security/MS16-015.

The following services within the listed versions of SharePoint are specifically affected:

1. Microsoft Office SharePoint Server 2007 (MOSS)

• Excel Services in SharePoint Server 2007 Service Pack 3 (32 bit edition)
• Excel Services in SharePoint Server 2007 Service Pack 3 (64 bit edition)

2. Microsoft SharePoint 2010

• Excel Services in SharePoint Server 2010 Service Pack 2

3. Microsoft SharePoint 2013

• Excel Services in SharePoint Server 2013 Service Pack 1
• Word Automation Services in SharePoint Server 2013 Service Pack 1

4. Microsoft Office Web Apps Server 2010 Service Pack 2

5. Microsoft Office Web Apps Server 2013 Service Pack 1

Background Summary (from Microsoft's Bulletin)

Full details on the vulnerabilities can be found here: https://technet.microsoft.com/library/security/MS16-015. According to the official Microsoft Bulletin the following is a summary of the vulnerability:

The most severe of the vulnerabilities could allow remote code execution if a user opens a specially crafted Microsoft Office file. An attacker who successfully exploited the vulnerabilities could run arbitrary code in the context of the current user. Customers whose accounts are configured to have fewer user rights on the system could be less impacted than those who operate with administrative user rights.

In addition, a cross-site scripting (XSS) vulnerability exists in SharePoint Foundation 2013 SP1 which could allow remote attackers to inject arbitrary web script or HTML via a specially crafted request.

The security updates provided by Microsoft address the vulnerabilities by:

• Correcting how Office handles objects in memory
• Providing a validly signed binary
• Helping to ensure that SharePoint Server properly sanitizes web requests

Security Resources

• VULNERABILITY DETAILS: All the information you need about this vulnerability and links to the required security patch can be found here:  https://technet.microsoft.com/library/security/MS16-015. 
• SECURITY UPDATES: Links to the security updates addressing all of these issues can be found at the link above, however a more direct link to the updates page for these security patches is the following: https://support.microsoft.com/en-us/kb/3134226.
• REPORTED EXPLOITS: According to Microsoft, at this time there are no reported exploits that have occurred using these vulnerabilities.

Additional details regarding the SharePoint related vulnerabilities are available at the National Vulnerability Database at the following links:

• Microsoft Office Memory Corruption Vulnerability – CVE-2016-0022 (Remote Code Execution via specially crafted Office document)
• Microsoft SharePoint XSS Vulnerability – CVE-2016-0039 (Elevation of Privileges via cross site scripting)
• Microsoft Office Memory Corruption Vulnerability – CVE-2016-0052 (Remote Code Execution via specially crafted Office document)
• Microsoft Office Memory Corruption Vulnerability – CVE-2016-0053 (Remote Code Execution via specially crafted Office document)

Security Strategy for Vulnerabilities

This bulletin reminds us that a comprehensive security strategy is needed for managing our server applications to ensure that we are alerted to critical security updates and we can make informed decisions about updating our servers to ensure that they are protected.  This is especially true when enterprises rely on SharePoint to store and manage sensitive corporate data. Sometimes these are managed through automatic updates.  In other circumstances, automatic updates are turned off on Production environments so that patches and updates can be tested in Staging environments prior to deployment to Production systems.  In many cases a mix of strategies is used, where critical security updates are automatically installed but other updates are not, so that they can be first tested in staging.  Which ever strategy your organization chooses, its important to identify one, ensure that its comprehensive and documented, and that it includes active periodic review of security updates on all server applications.

Personally, I'm not a fan of automatic updates.  I like to know what is getting installed on my systems, especially my servers - even when it comes to security updates.  But not having updates applied automatically requires active research or alerts so that we are informed when vulnerabilities are found and security updates are available.  I don't want to criticize automatic security updates - depending on your comfort level they are a viable strategy for managing security and protecting our servers from vulnerabilities.  I am a big fan of Microsoft's technical security notification service, which you can register for here:

https://technet.microsoft.com/en-us/security/dd252948.aspx

Once again, for the vulnerabilities discussed here, please refer to Microsoft's official bulletin for all details and required security patches which is located here:  https://technet.microsoft.com/library/security/MS16-015. 

eBook: Protecting Corporate Information from Insider Threats in Office 365

Enterprises face ever increasing and evolving threats to their internal corporate information every day.   The “insider threat” is just one of those threats, where risks come from internal employees exposing, stealing, destroying or over-sharing sensitive corporate data.  A threat from an insider presents unique challenges based on the fact that employees require legitimate access to data and systems in order to accomplish day to day work.  Some require privileged administrative access due to the nature of their role.  As well, employees typically have a need to know which information is most sensitive or valuable to the business.  Given that such access and knowledge is required to keep a business and its employees productive, how do we solve this problem in modern businesses today so that sensitive corporate information is protected, even from insiders?

Microsoft Office 365 provides an exceptional environment for hosting a corporate Enterprise Content Management (ECM) system, allowing all corporate information, both sensitive and non-sensitive, to be stored, managed and accessed in one place.  Microsoft has implemented excellent security measures, practices and assurances to protect organizations from external threats to their data.  It also provides useful tools to help organizations protect against internal threats.  Understanding the real challenges with internal threats will help enterprises make best use of those tools and put the most effective practices in place to protect the business from internal threats.

Click here to learn more and download the eBook:

Protecting Corporate Information from Insider Threats in Office 365

Webinar: What's New in SharePoint 2016

Thank you to everyone that attended our webinar on Thursday July 21 on What's New in SharePoint 2016!  We had a great turn out and some good questions.  There are a lot of new features and improvements in SharePoint 2016 and the ones I specifically spoke about in my webinar were:

1. MinRoles - Improving and simplifying SharePoint server deployments
2. Zero Downtime Patching - Improving how SharePoint is managed and kept up to date
3. Hybrid Search - Improving the end user experience when searching content in hybrid Office 365 and SharePoint Server environments
4. Other Hybrid Scenarios - Hybrid Sites, Hybrid OneDrive for Business, Hybrid Delve
5. User Experience Enhancements - Adding ease of use features such as the App Launcher, new library toolbar, new sharing capabilities
6. On Premise Compliance Center - Enforcing DLP policies and improved eDiscovery on SharePoint content
7. Removing and Improving Limits
• Improving List View Threshold Limits with Automatic Indexed Columns
• Improved Maximum File Size
• Removing File Name Character Limitations
• Site Collections per Content Database increases to 100,000
• Search Index Supports up to 500 Million Items

What Do We Lose with SharePoint 2016

Finally, we spoke about the features or capabilities which we lose in SharePoint 2016:

1. No new SharePoint Designer - but SharePoint Designer 2013 will work with SharePoint 2016
2. New SharePoint Foundation release
3. No SQL Express installed with single server deployments
4. No built in Forefront Identity Manager as part of the User Profile Synchronization
5. STSADM is officially deprecated
6. Excel Services is no longer part of SharePoint - it is installed with Office Online Server 2016 (the new name for the Office Web Apps product)

Questions

A few questions we answered during and after the webinar are:

• When is SharePoint 2016 expected to release?

Microsoft just released the RC (Release Candidate) build of SharePoint 2016 on Wednesday Jan 20th which is now available for free download and testing.  The RTM release is expected to release towards the end of the first half of 2016.

• Do you expect people to use the MinRoles feature when deploying SharePoint?

I think the MinRoles feature is a very useful feature for new SharePoint deployments which are straightforward.  However, but many complex SharePoint environments have been deployed with a variety of services for various reasons, many of which are specific to the client's specific needs.  I do suspect that many existing and complex deployments will use the Custom role and continue to customize the services deployed to their particular needs.

• Is InfoPath supported in SharePoint 2016?

Yes, Microsoft is still supporting InfoPath 2013 with SharePoint 2016.  Microsoft tells us that it will be supported into the early to mid 2020s.

• Since SharePoint 2016 no longer includes the Forefront Identity Manager as part of its installation to support the user profile synchronization process, what are our options here?

You essentially have 2 options here: 

1. You can use AD Import, which will be unidirectional (from AD to SharePoint) and will not right data back to AD
2. You can install the new Microsoft Identity Manager (MIM) yourself, which replaces Forefront Identity Manager, which will support 2 way synchronization as user profile synchronization currently does.

You can find the presentation deck from our webinar here: Webinar: What's New in SharePoint 2016.

What’s new in SharePoint 2016! from AntonioMaio2

Please let me know if there you have any other questions about this topic.

   -Antonio

My Reasons For Upgrading to SharePoint 2016

Excitement is brewing for the upcoming commercial release of Microsoft SharePoint 2016, which is expected to RTM in the first half of 2016. It’s already in its 2nd beta release with many users downloading it, installing it, providing feedback to Microsoft and providing the community with information about what’s new through blogs and articles.  Microsoft has provided some great new features that improve how SharePoint is managed, how we integrate with cloud services in a hybrid model and how we collaborate. 

It’s important to consider why we should upgrade a perfectly functioning SharePoint environment to this new major release.  Upgrading SharePoint is never just an upgrade.  We’re often looking at a migration, which can be a significant amount of work depending on the state of the current environment. Sometimes upgrades are driven by wanting access to some cool new features, but often the reasons are much more significant.  I'd like to share my favorite reasons for considering such a significant upgrade of SharePoint.

Hybrid Cloud Search

Hybrid search was introduced in SharePoint 2013 which would allow you to search across both your SharePoint on premise and your SharePoint Online environments through one interface.  It worked relatively well, but there were some caveats – most notably, search results from on premise sources and online sources appear separate and not integrated.  By default, search results are not merged and this can cause issues with determining which result is more relevant, paging of search results, etc.
SharePoint 2016 now provides a unified search experience for hybrid cloud search.  Essentially, the way this works is that the office 365 search functionality will consume the on-premise search index so that it can provide integrated results from both sources for the same query.  An Office 365 tenant and Office 365 search are required for this to work.  If users search using the SharePoint 2016 on premise search, only search results from local sources will be presented. There are other improved hybrid scenarios as well with SharePoint 2016, including:

• Hybrid Sites – Users can follow sites on both SharePoint on premise and in SharePoint Online and have them displayed in one unified list; users can also have a single profile in Office 365 where all their profile information is stored and kept up to date, and use that profile for both SharePoint on premise and SharePoint Online.
• Hybrid One Drive for Business – Users can sync files with Office 365; users can also access files directly through Office 365 on any device they might have.

If you are considering a hybrid ECM environment, with portions of SharePoint content stored on premise and portions stored in Office 365, the hybrid search experience available through SharePoint 2016 will certainly provide significant benefit to end users.

Zero Downtime Patching

Today, installing patches in a SharePoint environment can be a time consuming and disruptive process.  It requires the deployment of large 2GB+ cumulative updates on a somewhat regular basis.  All patches are currently distributed within 1 large update file which contains all patches up until that point in time including bug fixes, feature modifications and even schema updates.  In this form, all patches are applied at once, and results in some form of downtime to the environment.antonio1.13.162.png

SharePoint 2016 introduces a new concept for patching called Zero Downtime Patching, where patches are distributed through smaller packages (~100 MB) which may have some dependencies on one another so that certain patches are required before other patches can be installed.  This dependency model in fact already exists with cumulative updates.  The concept here is that patches contain smaller, more targeted changes so that, dependencies aside for a moment, administrators have some choice about which patches are needed and which they can skip, and so that patching with zero downtime is possible.

Applying patches on a regular basis is an important part of maintaining a healthy functioning SharePoint environment.  Doing so with more choice about which patches get applied and without downtime to the environment and its end users is an enormous improvement in the manageability of the overall SharePoint environment.

DLP Capabilities through the On Premise Compliance Center

SharePoint 2016 has directly integrated the fantastic DLP capabilities from Office 365 into its on premise version.  Traditionally we’ve had to look to third party tools to enforce DLP policies on SharePoint content, but with SharePoint 2016 this is now available through it’s out of box capabilities.  As well, SharePoint 2016 includes an improved eDiscovery Center which also allows you to query for content which do not conform to an organizations DLP policies.

SharePoint 2016 now allows administrators to create both a Compliance Center site collection and an eDiscovery Center site collection.  The Compliance Center site collection, much like the Office 365 Compliance Center, allows administrators to configure DLP policies which are automatically enforced on content within SharePoint.  Using the search service application, DLP policies will automatically identify content that is sensitive or confidential and flag that content so that it is not accessible by unauthorized users. These policies then provide some notification and remediation capabilities, like automatically notifying administrators and/or content authors by email of policy violations, and allowing overrides to policy warnings with documented justifications when needed.  In addition, like Office 365 the DLP capabilities come with 51 built in policy templates for identifying sensitive information.  These templates include a large number of regular expressions like credit card numbers and social security numbers, as well as regulatory compliance standards such as the PCI Data Security Standard and HIPAA related data.  The eDiscovery Center can now be used to run DLP queries to discover content which is sensitive or relates to industry regulations and export a report of the identified documents.

Most organizations use SharePoint to store and manage sensitive information in one form or another.  SharePoint 2016 now allows you to do so in a much more secure manager, and use the out of box capabilities to ensure that end users conform to corporate and industry regulations.

5000 List View Threshold Improvements

A common limitation we’ve had in SharePoint for some time is the 5000 threshold for the “List View Threshold” setting.  This is a limit (which can be configured, so it’s really a threshold) which determines how many items can be displayed in a single view.  This helps to ensure that performance of SharePoint overall is not adversely affected by users storing and displaying more that this number of items in a single view.  It ensures that queries to the SharePoint database for items in a list or library do not result in SQL database locks which will affect performance of the entire SharePoint farm.  Although this setting is configurable, we’ve always been recommended to stay away from increasing this past 5000 items.  One method of supporting large lists and libraries is to configure indexed columns on such lists.

Storing 5000 items in a list or libraries may sound large, but more and more organizations store a lot more content than this across many lists and libraries throughout their SharePoint portal.  Over time, it’s extremely easy to accumulate large numbers of lists which contain large numbers of items far beyond this 5000 limit.

SharePoint 2016 provides greater ability to manage large lists and libraries by now automatically creating indexed columns on these lists.  As content grows in our lists and libraries, which it inevitability will do, it’s important to ensure that the proper mechanisms are in place to efficiently manage that content, and SharePoint 2016 now automates more of that for us.

Large File Support

SharePoint 2016 now increases the recommended limit for storing single files from 2 GB to 10 GB.  This is a welcome change, as more and more businesses look to work with large files and large videos, for example presentations with large videos embedded within them, it’s important that the collaboration portal where such files are stored evolve with this changing need.

Microsoft Product Support

If you are currently running Microsoft Office SharePoint Server 2007 (MOSS) or Microsoft SharePoint 2010, Microsoft is no longer providing product support for either product. More specifically, Microsoft ended regular product support for SharePoint 2010 in October 2015. 

If you are in the situation where you are running either of these SharePoint versions, I would strongly recommend you consider planning your upgrade to SharePoint 2016.


In closing, there are many welcome additions and capabilities in Microsoft SharePoint 2016.  For me, the most compelling changes which will make me want to upgrade to SharePoint 2016 are those which improve SharePoint in broad ways from a management perspective, from an end user perspective and from a security perspective, which are those which we’ve outlined here:

1. Hybrid Cloud Search, allowing us to better support hybrid scenarios by providing better search results and ultimately a better user experience,
2. Zero Downtime Patching, enabling simpler management of the overall infrastructure and reducing downtime for users,
3. DLP Capabilities, allowing us to better secure the SharePoint environment and protect information containing sensitive information,
4. List View Threshold Improvements, allowing us to better support large lists and libraries through automatic indexed columns,
5. Large File Support, allowing us to better serve the needs of our end users as the files they work with get larger.

SharePoint Basics: MailTo Link with New Lines in the Email Body

I was recently asked about how to implement a MailTo link on a SharePoint site page, which is pretty straight forward.  The challenge for the person asking was about how to pre-populate the email body, so that it provided a template for users to fill out when sending that email, and how to have that email body contain multiple lines of text, with carriage returns and new lines between the lines.

There are several methods of doing this which do not work because SharePoint will actually remove the carriage returns and new lines.  There are other methods which do work. 

So, here is a quick post describing one method for accomplishing this.  I'm going to use JavaScript to setup the link so that the script is centralized in one place.

When including carriage returns and new lines in a link, we need to escape those characters so that web browsers can interpret them correctly. 

• The carriage return is escaped with %0D
• The new line is escaped with %0A
• If you want to have text start on one line, then move to the next line, you need to include: %0D%0A
• If you want to have an empty line between two lines of text, you would simply double the pattern: %0D%0A%0D%0A

In order to create a MailTo link which includes a pre-populated email subject, email addresses and body pre-populated with multiple lines, you can do the following:

• Create a JavaScript file which contains the following:

Click <a class="email" title="My Link Title" href="#" onclick="javascript:window.location='mailto:emailaddress@company.com?subject=Here Is My Subject Line&body=Here is the start of the email body %0D%0A%0D%0A email body continuing after a new line %0D%0A%0D%0A email body continuing after a second new line %0D%0A%0D%0A email body continuing after a third new line %0D%0A%0D%0A email body continuing after a forth new line %0D%0A%0D%0A email body continuing after a fifth new line.' + window.location;">here</a> to send an email template using a predefined template.

Notice how the link title field, email subject, email addresses and email body are populated.  Notice also how the carriage return and line feed characters are included in the email body.

• Upload the JavaScript file to the Site Assets SharePoint library
• On the site page add a Content Editor web part
• Edit the web part you just added
• In the Content Link property in the web part editor, add the path to the JavaScript file that you just uploaded to your Site Assets library.

Refresh your page and test your link.

   -Enjoy.

Securing Office 365 with Activity Monitoring

Thanks to everyone that attended my webinar last week on Securing Office 365 with Activity Monitoring.  We had a great turn out and the slides presented can be found here:

Securing Office 365 with Activity Monitoring from AntonioMaio2

Securing information systems is a very broad topic.  Monitoring and auditing these systems, and in particular the activities of users, is just one important aspect of securing our corporate IT environments. 

In July of this year, Microsoft announced some new capabilities around this within Office 365 – these are new Activity Monitoring and Reporting features.  These capabilities are designed to help organizations that are continually facing challenges with security, privacy and compliance.  In running and supporting the Office 365 service themselves, Microsoft has found that that they're capturing large amounts of data on which activities end users and administrators are performing.  They typically refer to this data as telemetry, and they've built great mechanisms into Office 365 to allow them to efficiently capture (and now share) this telemetry data.

These new capabilities provide greater visibility for administrators, and ultimately compliance and risk officers, into the actions taken by users on corporate content. They also allow us to apply greater access control over data, and if needed, they give us the capability to now investigate (at a very detailed level) user actions that might be against corporate or regulatory policies.

Why is Monitoring Activity and Auditing our Systems Important?

Monitoring user activity and auditing our information systems is important for many reasons.  

Regulatory Compliance

Regulatory compliance requirements are one key driver.  For example, many financial institutions often deal with MNPI, or Material Non-Public Information. Generally, this is information that’s not distributed to the public that an investor would likely consider important in making an investment decision.  Many institutions must put up Compliance walls to ensure that specific aspects of the business don’t communicate with each - this helps to avoid conflicts of interest and helps to ensure that they don’t inappropriately exchange MNPI.  

In particular, this is required in institutions which have both a corporate-advisory unit and a brokering unit, in order to separate those people giving corporate advice on takeovers from those advising clients about buying shares.  The wall is thrown up to prevent leaks of internal corporate information, which could influence the advice given to clients making investments

Detailed monitoring and auditing of user activity allows us to have a detailed view into which users are accessing sensitive content along who they’re sharing it with, and it provide assurances that our regulatory compliance obligations around in these business scenarios are being met.

Investigating Data Breaches

We've heard a lot about data breaches in recent years.  Data breaches can be small or they can be very large. They can be malicious or they can be accidental.   As well, data breaches can be caused by external actors like cyber criminals, or by insiders like system administrators or employees with broad levels of access.  Generally, we tend to see data breaches caused more often by external actors, but we see data breaches by insiders to involve larger quantities of data or more significant data. When data breaches do occur, it’s important for organizations to investigate and find the root cause so that they can both measure the scale of a breach (ie. how much data was leaked) but also to put in place measures to prevent these breaches in the future.  

When data breaches occur as a result of an insider threat, monitoring user activity at a detailed level allows us to perform investigations and root cause analysis to determine exactly who accessed data, when was it accessed and which actions were taken on that data - like who it was shared with.

Audit Access to Sensitive Information

In many organizations it’s important to audit the current access controls in place for sensitive content. This is sometimes referred to re-certifying permissions, or getting data owners to review and sign off that permissions are accurately set for data that they are responsible for.  In large organizations with large diverse information systems it can be really difficult to identify who is responsible for different data repositories.  

Monitoring user activity at a detailed level allows us to gain insight into who is accessing data on a regular basis, along with the level of access that they have.  This can greatly help us in identifying data owners to ultimately review and re-certify permissions.

Office 365 Activity Monitoring and Reporting

The new activity monitoring and reporting capabilities include:

• Office 365 Activity Report (built into the Office 365 experience)
• Comprehensive Event Logging
• Search PowerShell Cmdlet
• Management Activity API (in preview)

1. Office 365 Activity Report

You can access and run the Activity Report by:

• Logging into your Office 365 tenant
• Navigating to Admin in the App Launcher > Compliance Center > Reports > Office 365

[Activity Report Screen Shot]

You can use the Office 365 activity report to view detailed user and administrator activity in your tenant.  It contains data across SharePoint Online, One Drive for Business, Exchange Online and Azure Active Directory.  You can use this report to search and investigate user activities by searching for a user, a file or folder or even a site.  You can filter based on a date range or type of activity.  And within the report window you can view details of each activity in the Details Pane. The report is available to run on demand as needed.

When you find what you're looking for, you can either review activities and details right within this window or you can download the list of activities to a CSV file.

With each event captured there are up to 37 different properties logged.  Not all properties apply to all Office 365 services.  Some only apply to SharePoint Online and OneDrive for Business, whereas others only apply to Exchange.  The list of properties captured is shown here, with my favorites highlighted in red – my favorites are data like:

• Actor - The user that performed the action; can be a service principle
• ClientIP - The IP address of the device that was used when the activity was logged. The IP address can be either IPv4 or IPv6.
• EventSource – Identifies that an event occurred in SharePoint, OneDrive for Business or the ObjectModel.
• LogonType – Applies to Exchange only; this is the type of user who accessed an Exchange mailbox: mailbox owner, administrator, delegate, the Exchange Transport Service, a service account or a delegated administrator.
• Subject – Applies to Exchange only; this is the subject line of the message that was accessed.
• UserSharedWith – The user that a resource was shared with.
• UserType - The type of user that performed the operation: a regular user, an administrator in your Office 365 tenant or a Microsoft data center administrator.

You can see documentation on the full list of properties here:

https://technet.microsoft.com/en-US/library/ms.o365.cc.auditreporthelp.aspx#properties

2. Comprehensive Event Logging

In order to enable the Activity Report and make it really useful, events related to user and administrator activities are logged as users work across SharePoint Online, One Drive for Business, Exchange Online and Azure Active Directory.  

Currently there are over 150 events that are logged, and these are divided into 9 categories:

• Exchange admin events
• Exchange mailbox events
• File and folder events (SharePoint and OneDrive for Business)
• Invitation and access request events (SharePoint and OneDrive for Business)
• Sharing events (SharePoint and OneDrive for Business)
• Site administration events (SharePoint and OneDrive for Business)
• Synchronization events (SharePoint and OneDrive for Business)
• Azure Active Directory events (Admin Activity and User Login)

You can view documentation on the full list of events here:

https://technet.microsoft.com/en-US/library/dn708475.aspx

The events logged are diverse and very comprehensive, with Microsoft continually working to log more events.  When it comes to investigating data leaks, this gives administrators very detailed investigation capabilities to determine how leaks occur and how to prevent them.

3. Search Powershell Cmdlet

You can also search for events in the activity logs that we’ve been looking at using Powershell.  This is a new Powershell cmdlet to search all the event logs based on date range, the user who performed an action, the type of action, or the target object.

Examples of using this cmdlet are:

Search-UnifiedAuditLog -StartDate September 1, 2015 -EndDate September 30, 2015

Search-UnifiedAuditLog -StartDate 9/1/2015 -EndDate 9/30/2015 -RecordType SharePointFileOperation -Operations FileViewed -ObjectIds docx

With this capability we can script our searches of the event logs.  We can also have these searches output the results to a file.  And ultimately, this can allow us to schedule our reports to occur automatically on a regular basis so that administrators or infosec people can get insight into specific activities either every morning, every week or whenever the business schedule demands.

4. Management Activity API (in limited preview)

The final capability provided with this release is a new Management Activity API, which allows developers to integrate Office 365 activity and event data with either internal tools or with 3rd party monitoring and reporting solutions.

Full documentation on the Management Activity API can be found here:

• Getting Started - https://msdn.microsoft.com/EN-US/library/office/dn707383.aspx
• Management Activity API Reference – https://msdn.microsoft.com/library/office/mt227394.aspx
• Registering for the Limited Preview Program - http://dev.office.com/programs/managementactivityapideveloperpreviewprogram

There are a couple of important points about the API:

• This API is in limited preview now, and during the preview anyone can use the API, but only those registered with Microsoft will be able to actually retrieve data from Office 365.
• Actions and events are stored in content blobs in a database, and they are gathered across multiple servers and datacenters. As a result of this distributed collection process, the actions and events contained in the content blobs will not necessarily appear in the order in which they occurred. One content blob could contain actions and events that occurred prior to the actions and events contained in an earlier content blob.

Enjoy.

   -Antonio

Data Visualization Options in SharePoint and Office 365

A big thank you to everyone that attended my recent presentation last week in Houston on Data Visualization Options in SharePoint 2013 and Office 365.  We had a great turn out for our round table presentation with a lot of great dialog and questions.

You can view the presentation deck from our session here:

Data Visualization in SharePoint and Office 365 from AntonioMaio2

To summarize a few points from the session, Microsoft has several great data visualization tools available for SharePoint, including:

• Excel Services
• PowerPivot
• SSRS
• PowerView
• PowerBI
• Custom code with JavaScript
• PowerBI
• Datazen
• Performance Point
• Visio Services

Our presentation did not cover Performance Point or Visio Services due to the relatively low usage we see of those components.

Knowing which tool to use in which scenario can be really challenging.  So as part of the presentation we talked about the 3 questions you need to ask your self when choosing a tool, and we went through following use cases to help you decide the best tool for the job.  To recap that information...

The 3 questions to ask yourself when selecting a SharePoint data visualization tool:

• What do you want to do with your data? Do you want to create reports, create dashboards, data analysis, data discovery?
• Which devices are users consuming data on?  Are they using desktops, tablets, smart phones?
• Where is your data located?  Is your data on premise or in the cloud?

The various use cases we went through were the following, with our recommended data visualization tool.

Use Case 1

I am an Excel pro.  I have a lot of data. I have SharePoint on-prem… and I need to provide and share info to many users on Intranet

Recommended tool: Power Pivot

Use Case 2

I have SharePoint on-prem.  I want users to do data analysis and discovery on the intranet (SharePoint 2010/2013) on their own.

Recommended tool: PowerView

Use Case 3

I have SharePoint on-prem, and need to provide reports to business users on my intranet  which they will print.  I would like power users to be able to create reports.

Recommended tool: SQL Server Reporting Services (SSRS)

Use Case 4

I have both Office 365 and SharePoint on-prem, do not have Power View, cannot use my on-prem data in the cloud, but still need to do some kind of Visualization.

Recommended tool: Javascript code using standard Javascript libraries (D3.js, chart.js)

Use Case 5

I am using Office 365, have my data on-prem and want users to be able to use different devices to do data discovery on the data.  I want the users to do create these “reports”.

Recommended tool: Power BI

Use Case 6

I have on-prem SharePoint, the data is in lists and need to create responsive dashboards that work on many different devices.  I want my users to create and consume these.

Recommended tool: Datazen


Please let me know if you have any questions.
   -Antonio

How does Microsoft Protect Our Data in Office365?

I’ve received this question many times over the last year – clients who are considering Office 365 to store their corporate data asking:

How does Microsoft really protect our data as it sits within their data center?

Given the nature of my past security work, this is always a question that I’m happy to share the details about.  I often start by telling people that Microsoft has implemented an extremely robust, multi-layered security strategy for protecting data at rest in Office 365.  That sounds great, but what does that really mean?

Well, specific to SharePoint, OneDrive for Business and other solutions, Microsoft uses a multi-leveled encryption strategy with keys that are rotated (ie. regenerated) on a regular basis.  Actually the strategy is broader than that – it uses a combination of multiple levels of encryption, automatic key rotation, random distribution of data, drive level encryption and data spread across multiple systems each with their own network, OS, malware and physical protection.

In the on premise world, SharePoint data sits within content databases inside SQL Server.  You can certainly configure SSL communication between clients and SharePoint and between SharePoint and SQL to secure data in motion.  You can even enable Transparent Data Encryption (TDE) within SQL to secure your SharePoint data while at rest within the SQL Server database.  However, in the online world how exactly does Microsoft use encryption and other techniques to protect our corporate information?

Let’s look at how the strategy is applied in detail to your content within SharePoint and OneDrive for Business:

• Files within SharePoint Online and OneDrive for Business are shredded into fragments and each fragment is encrypted with a different key, using AES 256 bit crypto.  When files are modified, each delta is encrypted with a different key.
• Encrypted fragments are randomly distributed and stored across multiple Azure storage accounts.  These storage accounts are generated on demand and stored in separate systems.
• The keys used to encrypt fragments are regenerated once per day (key rotation).
• These keys are themselves encrypted using a master key that is specific to the customer.
• The master key is stored in a highly secured and monitored “key store” which is completely separate from SharePoint content databases.  The key store is the most secured asset in the Microsoft data center.
• The keys used to encrypt fragments, which are themselves encrypted with the master key, are stored in the SharePoint & OneDrive content databases along with a map to the fragments.
• Microsoft also uses BitLocker to encrypt all of the disks on all systems.

So let’s consider scenarios where the data center is attacked:

• If a content database is attacked, the attacker only gets access to a bunch of keys, which are encrypted and therefore unusable, and a map to the encrypted chunks that are stored in a different system with its own protections (the Azure storage accounts).  
• If the Azure Storage Accounts are attacked, the attacker only gets access to a bunch of random fragments, which are encrypted… and did I mention that the distribution of those fragments is random.  Even if they could decrypt the fragments, they will not be able to put a file back together due to the random distribution.
• Again, the key store is the most secure asset in the Microsoft data center.  Even if an attacker could get to the key store and attack it, at most they can get a key.  
• If the physical environment is attacked, and drives are physically removed and stolen, none of the data on the disk will be accessible due to BitLocker drive encryption.

Keep in mind all of the physical, network level, malware protections and internal procedures which strictly limit access to internal employees in the data center which are also in place.  In addition, Microsoft works every day to improve the security of their Office 365 offering by attacking and defending their own environments:

• They have a dedicated RED team, which sits outside of the Office 365 environment whose job it is to constantly attack the Office 365 environment looking for vulnerabilities and holes.
• They have a dedicated BLUE team which sites within the Office 365 environment whose job it is to constantly defend the Office 365 environment looking for ways to better protect our data from would be attackers.

Don’t those sound like the coolest jobs in the world?!

In The Future…

The next thing that Microsoft is working on to further enhance this strategy is to allow customers to bring their own master key, so that even if an insider wanted to access your data or a government request is made to Microsoft to access your data, Microsoft will not be able to retrieve it themselves.

You can find a great video on this topic here:  http://www.microsofttrends.com/2014/05/26/technical-details-on-office-365-fort-knox-encrypted-storage/.

As well, there was a great session at the Microsoft Ignite conference on this topic here:  https://channel9.msdn.com/Events/Ignite/2015/BRK3182.

Enjoy.
   -Antonio