Threat Hunting: Protecting Your Remote Workforce

VMware Carbon Black Cloud offers a converged set of security services to consolidate multiple endpoint security capabilities with a single lightweight sensor on endpoints, making it easy to add new capabilities whenever you need them.

Over the course of this guide, you will

  • Learn about the unique challenges facing security teams as workforces shift to working remotely
  • Learn how to think like a Threat Hunter and find new threats to your organization
  • Go hands-on with the VMWare Carbon Black Cloud and execute a threat hunt

Overview


  • Section 1: Accessing the Carbon Black Cloud Threat Hunting Experience
  • Section 2: Threat Hunting and Remote Workforces
  • Section 3: Accessing Carbon Black Cloud and Hands-on Threat Hunting
  • Section 4: Watchlists, Remediation, Live Query, and Behavioral Prevention

Before you Begin


In order to complete this product walkthrough please make sure you have the following:

  • A valid account in the VMware TestDrive environment, sign up here if you do not have one.
  • TCP & UDP ports 80, 443, 8443; and if using PCoIP, both TCP & UDP 4172
  • An Horizon Client installed on your machine.

 

Section 1:  Accessing Carbon Black Cloud Threat Hunting Experience

To login to the experience, perform the following steps.

First, open a web browser of your choice and navigate to portal.vmtestdrive.com and select LOG IN. If you do not already have an account please reference the instructions found here.

 

Next, locate the AppDefense product under the Intrinsic Security tab.

mceclip0.png

Click LAUNCH and LAUNCH VIA WORKSPACE ONE. If the LAUNCH button is grayed out, please click the Contact Us link in the Help & Support section.

A new tab will open with Workspace ONE. Enter your TestDrive Username, then hit Next.

mceclip0.png

On the next screen, enter your TestDrive Password then hit Sign in.

mceclip1.png

Next, search for the Carbon Black Cloud desktop. Click to open into the desktop either via HTML access or Horizon Client access.

mceclip0.png

Now you'll be on the Carbon Black Cloud RDSH desktop. At this point you can begin the walkthrough steps listed below.

mceclip1.png

 

 

Section 2: Threat Hunting and Remote Workforces

 

Section 3: Hands-on with Carbon Black Cloud

The VMware Carbon Black Cloud offers a converged set of security services and consolidates multiple endpoint security capabilities with a single lightweight sensor on endpoints, making it easy to add new capabilities whenever you need them. The CBC’s single, simple, cloud-based console is easy to access, configure, and use, with over 125 integrations with SIEM platforms, threat intelligence, and network security products that help you get more value from your existing security investments all while helping you operate faster and more effectively.

mceclip1.png

3.1  Accessing Carbon Black Cloud

On the desktop, launch the shortcut named “Carbon Black Cloud” (or open a Chrome browser and enter https://defense-prod05.conferdeploy.net).

Log in with the following credentials:

  • Username: vmtestdrive@vmware.com
  • Password: vmwareDemo1!

mceclip1.png

 

3.2  Dashboard

When you log in to the Carbon Black Cloud, you will first see a Dashboard containing a visualized snapshot of key information about your organization’s environment. The dashboard provides a high-level overview of your environment and enables you to quickly navigate to items of interest. You can customize the dashboard tiles and display data for specific time periods and policies. In the current dashboard setup we can see attacks that have been stopped by the CBC's advanced, behavioral NGAV, suspicious activity being monitored by behavioral analytics, endpoint information like health and VMware virtual machines, and the applications and machines that are creating the most alerts.

mceclip2.png

 

3.3  Threat Hunting and Building a Search

To begin our threat hunt, we'll start on the Investigate page.

Screen_Shot_2020-04-09_at_10.53.54_PM.png

For this walkthrough there is a pre-generated attack that we will hunt for and walk through. To ensure you see the attack, please set your timeframe to 1 month.

Screen_Shot_2020-04-09_at_10.56.15_PM.jpg

Once the time frame has been set, click the search Screen_Shot_2020-04-10_at_12.21.55_AM.png button. The CBC will automatically display all process activity within this organization for the last month. As stated in the video, a good time to hunt is anytime you hear about a new threat, so lets use the following mock threat report as a jumping off point:

Screen_Shot_2020-04-09_at_11.48.27_PM.png

This mock threat report gives us some great insight into how the attacker is accessing systems, so let's use that to find out if they have targeted any of our remote workforce. We can start by opening the Search guide by clicking the link just above the search button. In the Search Guide, click Search Fields. This shows all of the different process facets, activities, and metadata we can search for in the VMWare Carbon Black Cloud. Take a moment to explore some of the fields like ProcessParent Process, and Network Connections.

Screen_Shot_2020-04-09_at_11.38.21_PM.jpg

To begin our search, we'll build a query based on the information given to us in the mock threat report. In the report, the attacker is said to use macro-enabled Word documents. We can use the "process_name" search term to narrow in on any execution of Word in our environment. To do this, enter "process_name:winword.exe" into the search bar and click the search Screen_Shot_2020-04-10_at_12.21.55_AM.png button. As you type, notice that the search bar automatically begins to suggest terms that match what you have entered so far.

Screen_Shot_2020-04-09_at_11.53.37_PM.png

Clearly, there are far too many results for us to narrow in on any one of these executions of winword.exe as being malicious. The mock threat report also stated that the attacker sends their malicious documents through phishing emails. To further narrow our search, we can make use of the CBC's search operators to look for any instance of Word being opened by an email client, such as Outlook. In the search bar, add the operator "AND" followed by "parent_name:outlook.exe" and click the search Screen_Shot_2020-04-10_at_12.21.55_AM.png button.

Screen_Shot_2020-04-10_at_12.23.39_AM.png

This query has drastically narrowed our results. If this search was still too broad, we could continue to refine it by looking for network connections, ports, and other activity mentioned in the mock threat report. However for our current purposes, this is enough. We'll cover how to use the other information from the threat report later.

To continue our hunt and find out exactly what this rogue instance of Word did, click on the process name winword.exe in the results. 

Screen_Shot_2020-04-10_at_3.03.47_AM.jpg

3.4 Navigating the Process Analysis Tree

This is the Process Analysis page. On this page, we can walk backwards and forwards through the timeline of events and processes as this attack was executed. The currently highlighted process node is for winword.exe, and we can see that winword.exe was opened by outlook.exe.

Screen_Shot_2020-04-10_at_12.38.00_AM.png

To the right of the Process Analysis Tree, all of the information and metadata for winword.exe is displayed, including the command line, user context, and hash values; the process reputation; and information about the process's access controls. Click the help Screen_Shot_2020-04-10_at_12.44.18_AM.png button next to the Reputation and Process Access Control text to learn more.

Screen_Shot_2020-04-10_at_12.39.13_AM.png

Below the Process Analysis Tree are the event filters for the currently selected process node. Facets like registry and file modifications, network connections, child and cross process events, and much more are all recorded and searchable.

Screen_Shot_2020-04-10_at_12.41.50_AM.png

While looking at the Process Analysis tree, to see what happened after winword.exe executed, click the plus sign on the winword.exe process node to expand the timeline. The tree shows that winword.exe executed rundll32.exe, and we can click the plus sign on the rundll32.exe node to expand its events.

Screen_Shot_2020-04-10_at_12.49.15_AM.png

Now the attack starts to truly take shape. The malicious Word document contained a macro that executed powershell.exe via the rundll32.exe process.

Before we move on to look at what Powershell executed, let's look at the process filters for rundll32.exe. Scroll down and use the netconn filter on the left side of the page to look at any network connections rundll32.exe made.

Screen_Shot_2020-04-10_at_12.54.37_AM.jpg

In these results, a TCP connection to the IP address 129.146.208.70 was established over port 443, the same port listed in the mock threat report as being used for primary command and control. If this were a real threat actor in our environment, this is crucial information that we would use to tighten the network security controls at our endpoints.

Now, return to the Process Analysis Tree and click the plus signs on the powershell.exe process nodes. 

Screen_Shot_2020-04-10_at_1.01.18_AM.png

Look for the Powershell node that executes schtasks.exe as shown in the screenshot above. To the right of the tree, in the details for this instance of Powershell, you can see that Powershell executed with an encoded command.

 Screen_Shot_2020-04-10_at_1.03.40_AM.png

To de-obfuscate this encoded command and see it in plain text, scroll down to the event filters and filter by fileless_scriptload. To see the entirety of the decoded command, expand the event and click the button next to the truncated command string.

Screen_Shot_2020-04-10_at_1.06.01_AM.jpg

Let's return to the Process Analysis Tree and continue to explore the different instances of powershell.exe and what they executed. Click on the process node for schtasks.exe. In the process information to the right, the command line information shows that a task named spawn was created.

Screen_Shot_2020-04-10_at_1.13.27_AM.png

Now, find the Powershell process node that invokes another instance of powershell.exe, and expand the process tree by clicking the plus sign. This new instance of Powershell was used to execute whoami.exe and lsass.exe. LSASS is where password and login information is stored on an endpoint, indicating the attacker was attempting to steal any credentials that may exist on this machine.

Screen_Shot_2020-04-10_at_1.17.40_AM.png

Now that we've explored this attack, we can do several things. We should make sure that we are alerted if this ever happens again, we should find out if any artifacts of this attack exist anywhere else in our environment, and we need to take remediative actions to stop this attack from spreading further and undo any change the attacker made at the affected endpoint. Some of these steps are not possible with the credentials you used to log in to the CBC console, so we will walk through them in a video after the next section. For now, lets start by taking what we've learned in this threat hunt and using it to build a better, more repeatable search based on the pattern of activity we just saw.

3.5 Building a Pattern-based Search

In our initial threat hunt, we used the exact description of the attacker's activity from the mock threat report. This was adequate to find this attack in this exercise, however the attacker could easily change pieces of their attack in the future. Word isn't the only office application that can run macros, and Powershell isn't the only command interpreter the attacker could use. To build a search that focuses on the pattern of this attack instead of the exact sequence of events, return to the Investigate page. 

To begin, lets focus on the attacker's initial execution through Word. While we could use the CBC search operator "OR" to string together every application capable of running macro-enabled documents, it will be much faster to use wildcards. Start by typing "parent_name:*\office15\*" into the search bar and clicking the search Screen_Shot_2020-04-10_at_12.21.55_AM.png button. 

Screen_Shot_2020-04-10_at_1.47.20_AM.png

This search differs from our initial search in two key ways. First, instead of looking for the specific process of "winword.exe", we have used the wildcard * in conjunction with a folder structure. The leading * ensures that the search is not looking for specific user folders, and the trailing * ensures that we get results for any application or process inside the Office15 folder. Second, and more importantly, we are now using the "parent_name" term instead of the "process_name" term. By using the "parent_name" term, all our results will be processes that were either created or opened by an application in the Office15 folder. In the results, you can see instances of both winword.exe and rundll32.exe, both of which we saw in the hunt we just conducted.

Now, let's add some more detail from our hunt. In the attack we walked through, the attacker used port 443 for command and control communication. To add this to the search, use the "AND" operator and "netconn_port:443".

Screen_Shot_2020-04-10_at_1.57.40_AM.png

Using common ports like 443 is a technique used to blend in with normal, day to day web traffic. If you were simply to search for any connection made over port 443, you would find far more uneventful data than you would find attacks. However, by pairing this port with the previous "parent_name" term, we have narrowed the results to only process trees that start with an Office application and result in a network connection. This is still fairly broad though, so we will add the final pieces of the puzzle.

Once the attacker established C2, they began executing a series of commands via Powershell. However, they could have also used another terminal application, like cmd. To complete our search, we'll use the "AND" operator, followed by "process_name:cmd.exe OR process_name:powershell.exe". Now, no matter which command interpreter the attacker might use, this search will return find their activity.

Screen_Shot_2020-04-10_at_2.05.29_AM.png

In this form, this search takes into account the various ways the attacker could alter their attack structure, while still being specific enough to return actionable results. Click the search Screen_Shot_2020-04-10_at_12.21.55_AM.png button to see what kind of results this new search finds.

The remaining steps we should take when conducting a threat hunt aren't available to you in this course. Watch the videos below to learn about creating alerts by turning a threat hunt search into a Watchlist, executing real-time queries to see information about your endpoints' current state, and taking decisive, surgical remediation actions to stop attacks and undo change.

Section 4: Creating Watchlists, using Live Query, and remediating through Live Response

4.1 Using Watchlists

Watchlists are an extension of the threat hunt search you just built. Once you have found malicious activity in your environment through a search, you can save it as and Indicator of Compromise (IOC) in a Watchlist. Watchlists run continuously, analyzing the recorded data from your environment as it is sent to the VMWare Carbon Black Cloud. If a sequence of events matches an IOC, the matching process is flagged in the console. Watchlists can also generate alerts to notify you immediately if they are triggered. To see how a Watchlist is created, and how VMWare Carbon Black delivers actionable threat intelligence directly to our customers' consoles through Watchlists, watch the video below.

 

4.2 Using Live Query

Live Query is a powerful, SQL-based querying system built into the Carbon Black Cloud sensor. With Live Query, you can create queries to retrieve current, stateful data about your protected endpoints and servers. From the Live Query page, users can run pre-defined queries created by VMWare Carbon Black, find queries submitted by our user community, or build their own queries to meet their specific needs. Queries can be executed immediately, or scheduled to run at regular intervals. Using Live Query you can find artifacts of attacks that may have occurred prior to installing the Carbon Black Cloud sensor, quickly identify endpoints or servers that are misconfigured, and find areas for improved security and IT hygiene.

 

4.3 Remediating through Live Response

Live Response is built into every aspect of the VMWare Carbon Black Cloud platform. Whether you are triaging alerts generated by the behavior-based NGAV, investigating new attacks through threat hunting, or finding security misconfigurations with Live Query, Live Response allows you to directly interact with your endpoints and servers. Using Live Response puts you in direct control of a machine, no matter where it is in the world, and gives you full access to upload and execute scripts, retrieve or delete files, and run any program or function native to that endpoint or server. In addition to Live Response, the CBC has single-click remediation options like file deletion or network quarantine. To see how to use Live Response and respond to incidents in real time, watch the video below.

 

 

For Additional Support

Learn more about VMware Carbon Black Enterprise EDR

Contact VMWare Carbon Black Sales

Have more questions? Submit a request

Please sign in to leave a comment.