A Practical Overview of Office 365 Advanced Security Management - Part 2Productivity App Discovery Dashboard

In the middle of 2016, Microsoft released the first version of Office 365 Advanced Security Management, a new capability within the Office 365 platform that allows organizations to go above and beyond the typical security management features, helping them to better secure users, permissions, content and apps. This multi-part blog series will look at how to use the features that make up Advanced Security Management (ASM) and share technical details that will help you to understand the benefits of these robust tools. In part 1, we introduced Advanced Security Management and shared technical information about how it works with the Office 365 Unified Audit Log: A Practical Overview of Office 365 Advanced Security Management - Part 1. In part 2, we review the Productivity App Discovery Dashboard capability of ASM to see how log files are imported, how to create reports and review the results of ASM's analysis of those logs, and how you can try it out with some built-in sample logs.

Upon launching the Office 365 Advanced Security Management console, one of the new capabilities available is the Productivity App Discovery Dashboard page, which was added to ASM around October 2016. You can access it through the ASM menu by clicking Discover > Discovery Dashboard:

The dashboard allows you to quickly review security reports that ASM generates from log files that you upload to ASM from your firewalls, proxies and other network security devices. This capability is really targeted at analyzing logs from security appliances and perimeter devices or applications. Once you’ve uploaded at least one log file and generated a snapshot report, there is a lot of useful security data provided in an easy to view dashboard:

Creating a New Snapshot Report

In order to upload a log file from one of your network security appliances you must use a manual process. You need to retrieve the log file from your appliance for some particular time period, log into the Office 365 Advanced Security Management console and then do the following:

• In the ASM menu shown above, click Discover
• Click +Create New Report (see screenshot of the menu above)

The following page will appear:

Now give your new report a Name and Description. Then select the data source, or the type of appliance from which the log file was retrieved:

In this list there are in fact 20 network security solutions to choose from, from 13 vendors. Each solution vendor or device may have a different log file format which will dictate how the logs are analyzed. The vendors and appliances listed are:

• Blue Coat ProxySG - Access log W3C
• CheckPoint
• Cisco ASA Firewall
• Cisco FWSM
• Cisco IronPort WSA
• Cisco ScanSafe
• Cisco Meraki - URLs log
• Dell SonicWALL
• Fortinet FortiGate
• Juniper SRX
• McAfee Web Gateway
• Microsoft ForeFront Threat Management Gateway (W3C)
• Palo Alto PA Series Firewall
• Sophos Cyberoam Web Filter and Firewall log
• Sophos SG
• Squid Common
• Squid Native
• Websense Web Security solutions - Internet Activity Log (CEF)
• Websense Web Security Solutions - Investigate detail report (CSV)
• Zscaler

...and then there is an Other (unsupported log format) option as well, if you have some other solution in your network.

Report Processing
Once you've selected your solution type, you click Browse, select your log file and upload it for processing. You may upload multiple log files at once. Once uploaded, ASM will process the file to analyse traffic going through your network security solution and produce the Discovery Dashboard. This process can take a few minutes, but it will let you know once that processing is complete.

Report Ready
Once complete, your report will show a status of Ready and you'll be able to view the analytics and insights provided by the dashboard.

Limitations
Note: There are some limits on the log files that ASM will process that are important to remember:

• Each log file may be up to 1 GB in size.
• Uploading a log file is an entirely manual process. There is no PowerShell available at this time to start the upload, nor to access any ASM functions. So the log file upload process cannot be automated unfortunately.
• You may only have 10 reports at a time. If you already have 10 and try to create an 11th, you will be told to first delete one of your existing reports.
• As your report ages, any entries that are older than 90 days will be removed from the report and dashboard results. This time period cannot be adjusted at this time.
• Entries in log files that are older than 90 days will be ignored. If your entire log file is older than 90 days, ASM will still attempt to process it upon upload but the result will be an expired report which you will not be able to view, as shown here:

Analysing and Categorizing Cloud Apps

When log files are analyzed by ASM, the traffic is not only categorized by the type of cloud app (like cloud storage or social media), but ASM will recognize the individual cloud apps and let you know exactly which ones your end users are accessing. The larger Cloud App Security (CAS) solution has a catalog of almost 15,000 cloud apps that it will identify. However, a subset of those apps are recognized as 'productivity apps' and it is that subset of over 1,000 apps that ASM will identify. This list is actually a growing list, and Microsoft has a team of people working on increasing it through both manual and automatic methods. Microsoft is committed to support new apps in their catalog as soon as possible.

If you have a cloud app that you know is in use and is not yet recognized by ASM, you can request that Microsoft investigate it and potentially add it to the catalog. This is typically done by opening a support request through the ASM portal (question mark on the top right of the page). When creating a service request, you'll need to select "Cloud App Security" under the Create a Service Request list.

Analyzing and providing a quick view into which cloud apps are in fact in use by end users can really help you to determine if Shadow IT is at play in your organization, and exactly how much data is moving to cloud hosted solutions which may not be IT approved. This can provide great visibility to IT and InfoSec teams, helping them to work with end users and business units to ensure they are using the corporate approved solutions and not exposing the organization to risk.

Sample Log Files for Trialing the Discovery Dashboard

Another great feature that's built into the Discovery Dashboard is that it provides sample log files for you to trial and learn about the insights this feature can provide. You can access the sample logs by following the process described above for creating a new snapshot report, and when you get to the page where you give your report a name and description, select the type of Data Source from the dropdown and then click the "View and verify..." link:

You'll then see a page that describes the log format required by that solution in some detail. On that page, click the "Download sample log" button. You'll download a log file to your desktop, which you can then use to create a sample snapshot report.

The log file will download as a ZIP file, which you'll have to save locally, unzip and then start the process once again to create a snapshot report. The log file downloaded will follow the proper format for the Data Source selected (the solution vendor and type). So follow the process outlined above to create a new snapshot report once again using the sample log and choose that same data source the next time through.

These sample files are updated each week so that they stay fresh and do not result in expired reports (older than 90 days).

A Tour of the Productivity App Discovery Dashboard

Once you've created a snapshot report, as described above, you can access the Discovery Dashboard to get an overview of the analysis performed on our log files.

Selecting a Report
You can select which snapshot report you're viewing on the top right corner of the screen:

Traffic Statistics
On the top left of the Discovery Dashboard we're presented with statistics about:

• number of apps analyzed
• number of users referenced within my network security solution's log file
• number of IP addresses analyzed
• amount of network traffic analyzed

Remember, these statistics are only from log file entries that are not older than 90 days.

As you can see, the amount of traffic that went through the network security solution is shown, and its broken up by the amount of traffic uploaded (red arrow) versus downloaded (black arrow).

Cloud Apps and Categories
Moving a little further down the page on the left side, we can see the categories of cloud apps and the apps that are themselves in use. In our top table on the left, ASM has categorized the cloud applications found in my log file into the categories shown here. This gives us a quick view into the type of cloud apps in use by users, or the capability which that cloud app offers to users. It also shows the total amount of data transmitted through that cloud app. For example, in this case we can see that 2.3 GB has been exchanged with cloud storage solutions (Box, Dropbox, etc.) and only 4 MB has been exchanged with social media cloud based apps (Twitter, Facebook, etc.).

On the bottom table on the left, we see the actual cloud apps that are most in use in my organization along with the amount of traffic generated for each (in this case OneDrive for Business, Box, Skype for Business, Office 365 and Exchange). However, I can see right beside the "Discovered apps" label that there were in fact 116 different cloud apps found going through my network device. I can use the dropdowns above that table to view the other categories of cloud apps as well. So if we select social networks from the list, I see the apps that found in my log which fall into that category, no matter how small the amount of data.

As well, you can easily include or exclude Office 365 traffic from these graphs by unchecking or deselecting "Office 365". So, if Office 365 is a corporate standard collaboration solution this allows you to easily focus on other cloud apps which may not be approved.

Risk Levels, Traffic Locations and Exporting Data
Finally, in the diagrams on the right side of the page, ASM provides a risk score for the traffic analyzed and a map of where the traffic is originating. We can see in the following that after analyzing 2.7 GB in total that 4 KB is considered high risk traffic, 83 MB is considered medium risk and 2.6 GB is considered low risk. I can see these numbers by hovering over each slice of the pie. I can also see a world map in this section of the dashboard which gives me a quick view into where the traffic going through my network is originating from - the map is generated by the IP addresses found in my log file.

The risk level shown for the various types of traffic is based on intelligence and heuristics that ASM uses from the Microsoft Security Graph in its analysis to determine if risky IP addresses or non-reputable cloud apps are being accessed through your network. This integration of intelligence from the Microsoft Security Graph is one of the major benefits of ASM over the other built-in security tools in Office 365. You can learn more about Microsoft's intelligent security graph here: https://blogs.microsoft.com/microsoftsecure/2016/07/21/new-microsoft-azure-security-capabilities-now-available/.

If I want to review the specific risk details by clicking "View risk details" we are unfortunately told that I need the Cloud App Security solution to do that:

I can adjust this graph through the dropdown to instead focus on apps, Apps, Users, IP Addresses, Upload Traffic and Transactions. On each graphic on the dashboard, I can also click the little grey downwards arrow to download a CSV file of the traffic details shown in the graph. The CSV file is relatively simple but can be useful when we need to generate reports for others in the organization.

Automatic Log File Upload and Cloud App Security

The Productivity App Discovery Dashboard is a great solution for analyzing traffic going your network and understanding which cloud apps end users are making use of. It can provide IT and InfoSec teams with the information they need to determine if Shadow IT as at work in their organization, and give them intelligence they can use to work with end users and business groups to ensure they're following corporate policy for cloud based collaboration.

As mentioned above, the upload of log files is a manual process and we're limited to having 10 reports in the dashboard at a time. I think this can still be a beneficial solution to enterprises in circumstances where you want to perform adhoc analysis a firewall or network security log, as part of a regular IT security audit or when investigating suspicious network activity. If you want to make ongoing use of this capability however and have log files uploaded automatically then for now you'll need to upgrade to the larger Cloud App Security solution from Microsoft. You can learn more about Cloud App Security here What Is Cloud App Security, and I may also cover it more in a future post.

What's Next

This post was a fairly thorough review of the Productivity App Discovery Dashboard feature within Office 365 Advanced Security Management.

The next post in this series will look at how security policies and alerts are in fact configured and how they work within ASM.

Enjoy.
-Antonio