Securing Modern Applications with CBC Container Security

About this experience


Throughout this walkthrough you will be exposed to VMware’s solution to securing Kubernetes clusters with CBC Container Security. Upon login, instructions provided will detail how to deploy and secure a new cluster with vSphere with Tanzu and CBC Container Security.

What is CBC Container Security?

CBC Container Security helps organizations reduce risk, obtain compliance and achieve simple, secure cloud-native Kubernetes environments at scale. With a simple, no-friction deployment process, this solution provides the visibility and control that Development and Security teams need to secure Kubernetes clusters and the applications deployed on them. Harness the power of Carbon Black Cloud Container for your build and deploy pipelines, with instant visibility into all your Kubernetes workloads, and the ability to enforce compliance, security, and governance from a single dashboard.

For more best practices on Carbon Black Container Security please visit VMware Carbon Black TechZone here.

What is vSphere with Tanzu?

You can use vSphere with Tanzu to transform vSphere to a platform for running Kubernetes workloads natively on the hypervisor layer. When enabled on a vSphere cluster, vSphere with Tanzu provides the capability to run Kubernetes workloads directly on ESXi hosts and to create upstream Kubernetes clusters within dedicated resource pools.

By using vSphere with Tanzu you can turn a vSphere cluster to a platform for running Kubernetes workloads in dedicated resource pools. Once enabled on a vSphere cluster, vSphere with Tanzu creates a Kubernetes control plane directly in the hypervisor layer. You can then run Kubernetes containers by deploying vSphere Pods, or you can create upstream Kubernetes clusters through the VMware Tanzu™ Kubernetes Grid™ Service and run your applications inside these clusters.

What is Tanzu Mission Control (TMC)?

VMware Tanzu Mission Control™ is a centralized management platform for consistently operating and securing your Kubernetes infrastructure and modern applications across multiple teams and clouds.

Available through VMware Cloud™ services, Tanzu Mission Control provides operators with a single control point to give developers the independence they need to drive business forward, while ensuring consistent management and operations across environments for increased security and governance.

Table of Contents


Before You Begin


Before you begin this walkthrough please make sure you have the following:

1. A valid Cloud Services account in the VMware Pathfinder environment

2. If using the Horizon Client, ensure it's installed on your machine and that the following ports are open TCP & UDP ports 80, 443, 8443; and if using PCoIP, both TCP & UDP 4172 on your network. You may also access this walkthrough on Horizon via the Browser.

3. When using the walkthrough, please do NOT create more than 1 Tanzu Kubernetes Cluster

 

Also, we encourage everyone to use the My Documents folder located on the VMware Tanzu Horizon Desktop to store yaml files and container images. This folder is persistent which means that users can retrieve their stored application data across multiple sessions.

Logging Into CBC Container Security Demo

To access the CBC Container Security demo perform the following steps.

First, open a web browser, navigate to https://pathfinder.vmware.com and log in with your Cloud Services account. If you do not have an account, click on 'Create your own VMware Account' option on the Pathfinder login screen.

Screen_Shot_2021-05-23_at_12.14.34_AM.png

Once logged in, navigate to 'CATALOG' from the top navigation bar, and choose 'Carbon Black' as the product, 'TestDrive' as the Activity Type as shown below.

KB_Activity_CBC.png

Click on the 'Get Hands-on with VMware Carbon Black Cloud Container Security' Activity Card, the activity opens up. On the Getting Started page, click on 'Request Access' button located near the bottom-right of this page. This will provision your TMC and vSphere with Tanzu account and display 'Tanzu Missionn Control' and 'Workspace ONE' credentials as shown below. Please note that both your TMC and vSphere with Tanzu accesses are valid for 48 hours in a single session. Once the time expires, you can re-enable access by following the same method.

PF_Access_Step_5.png

Next, launch the 'Workspace ONE' link under Access option, you will be redirected to Workspace ONE Access login screen. Use the Workspace ONE Credentials from the Credentials section.

mceclip3.png

Once logged in, search for 'VMware Tanzu' on the Apps Tab and click on the Horizon Desktop to launch it on your Browser. To launch the Desktop using Horizon Client, click on the Info (i) button and select the 'Client' option.

mceclip1.png

Now you'll be on the VMware Tanzu Desktop. At this point you can begin the walkthrough steps listed below.

 

Section 1: Quick Walkthrough of Carbon Black Cloud


1.1 Login to Carbon Black Cloud

Launch the Carbon Black Cloud shortcut on your desktop as shown.

mceclip2.png

Login using the credentials listed in Demo Credentials.txt file located on your desktop.

mceclip3.png

  • Username:  (listed in text file Demo Credentials.txt on the VMware Tanzu desktop)
  • Password:  (listed in text file Demo Credentials.txt on the VMware Tanzu desktop)

The Carbon Black Cloud dashboard provides a high-level overview of your environment and enables you to quickly navigate to items of interest. You can customize the dashboard tiles and display data for specific time periods and policies.

mceclip0.png

 

1.2 Reviewing K8s Inventory

For this walk through you will have the ability to build a new Tanzu Kubernetes Cluster and deploy CBC Container Security in sections 2 & 3 of this walk through.

Navigate to Inventory > Kubernetes > K8s Clusters to review current cluster management.

CBC Inventory provides Security teams with unified insights across Endpoints, Workloads and Containers. Security teams are able to view all protected and active clusters running within their enterprise.


mceclip6.png

Expand the > under Actions column to review cluster details.

mceclip7.png

To assess K8s Workloads running within your environment navigate to Inventory > Kubernetes > K8s Workloads. 

K8s Workloads | K8s Workloads monitor and provide information for each workload. You can view risk severity, if a workload is covered by a K8s policy, and if there are any policy violations. This helps remediate risks and fix issues at a workload level in your K8s environment.

mceclip8.png

 

1.3 Kubernetes Health

To understand your current state of you K8 Workloads navigate to Harden > K8s Health

Kubernetes Health Overview: Provides a single pane of glass for complete visibility into your security posture across k8s clusters and namespaces, including visibility into rules violations and configurations. The K8s Health page shows the current state of your Kubernetes environment and a summary on potential vulnerabilities. The vulnerabilities are split into four categories: Workloads, Network, Operations, and Volume.

  • Workloads group built-in rules which identify settings that may expose your deployment to attack.
  • Network groups built-in rules which identify Ingress services (read more for Ingress) in use in your deployment.
  • Operations group built-in rules which identify performance and utilization of workloads.
  • Volume groups built-in rules which identify access to data within your deployment.
  • Container Images groups built-in rules which identify issues and vulnerabilities within your container images.
mceclip9.png

 

Next, let's switch the Tab from Overview to Risks. For more details on a particular risk, highlight it by clicking on the right hand arrow '>'

mceclip10.png

Kubernetes Health Risks: The K8s Health page displays identified risks and associated workloads after setting up your Kubernetes clusters. Reduce risks in your K8s environment and create policies to enforce Alert or Block actions in the future if any rule validation fails. The K8s Health page will reflect all changes in your environment once you have taken action to resolve any potential vulnerabilities. Risk is defined by KCCSS which scores risk based on:

  • Confidentiality: Exposure of PII (Personally Identifiable Information), potential access to secrets, PII, etc.
  • Integrity: Unwanted changes to the container, host or cluster such as being able to change the runtime behavior, launch new processes, new pods, etc.
  • Availability: Exhaustion of resources, Denial of Service, etc.

Section 2: Protect a K8s Cluster with VMware


To get hands-on with the VMware Cluster and Security Deployment, we will:

    • Deploy our Tanzu Kubernetes Cluster
    • Configure CBC Container Security
    • Enfore Pod Security Standards using CBC Container Security

2.1 Deploy Tanzu Kubernetes Cluster

On your VMware Tanzu Desktop, launch the 'Tanzu Mission Control' Chrome Shortcut

mceclip2.png

You will be redirected to the VMware Cloud Services login page. Use your email address associated with your VMware ID account, followed by your VMware ID password. You can look at the Credentials section below to find your VMware ID.

mceclip3.png

Once logged in, you should see a My Services page with 'VMware Tanzu Mission Control' listed as a product in it. Launch the product.

mceclip14.png

If you don't see it, then click on your username on the Cloud Services page -> Change Organization and make sure you have selected 'Pathfinder Services' as your Organization. After that, you should see the TMC product.

TMC_change_org.png

Now that you're logged in, you should see the 'Clusters' page as your default landing page. We will create a Tanzu Kubernetes Cluster next. Click on 'Create Cluster'

mceclip4.png

On the next screen that comes up, click the radio button next to pathfinder-tanzu and 'Continue to Create Cluster'

mceclip5.png

Choose the provisioner as pathfinder-tanzu from the dropdown and click 'NEXT'

mceclip6.png

Let's name your cluster. Use the format <your-username>-tanzu for your cluster name. 

Next we must click on Cluster Group, by default you will see "pathfinder-tanzu-demo". Click on the 'X' to delete this entry value

Now click the drop down to select the Cluster Group name that should be auto-populated with the name formatted as <your-username>###. Click 'NEXT'

Note: Tanzu will not let you finish creating your cluster in the following steps if you leave the Cluster Group name as the default value of "pathfinder-tanzu-demo" 

mceclip11.png

Under the configure section, select your desired Kubernetes version and click 'NEXT'.

PLEASE NOTE that the highest supported version for this demo is v1.18.5. You may want to select 'vsphere-with-tanzu-storage-policy' as your persistent volume storage (Optional).

mceclip10.png

On the 'Select Control Plane' page, use Single Node with Instance Type as 'best-effort-small (2vCPU, 4GB RAM)' and click 'NEXT'.

NOTE: Since this is a shared demo environment, we request all users to follow the guidelines for selecting Instance Types to optimize resource usage.

mceclip9.png

On the 'Edit and Add Node Pools' page, select the Worker Instance Type as 'best-effort-small (2vCPU, 4GB RAM)' and number of worker nodes = 1 (default) and click on 'CREATE CLUSTER'

NOTE: Since this is a shared demo environment, we request all users to follow the guidelines for selecting Instance Types to optimize resource usage.

mceclip12.png

You will now see a screen with message 'Your cluster is being created'. Please allow 5-7 minutes for the Tanzu Kubernetes Cluster to show as 'Ready'

mceclip13.png

Once your cluster is ready, click on 'Actions' at the top right corner and choose 'Access this cluster'

mceclip15.png

A new dialog box will open. Click on 'DOWNLOAD KUBECONFIG FILE' and Save As 'config.yml'

NOTE: It is important to save the file as config.yml in order for the next set of steps to work.

mceclip16.png

Launch Windows PowerShell from the Desktop by double clicking on the shortcut. Set the KUBECONFIG environment variable to point to our config.yml file saved in the previous step by copying the below command and pasting it in PowerShell.

$env:KUBECONFIG = "\\vmwtd.com\tdpublic\UEM-REDIRECT\$env:USERNAME\Downloads\config.yml"

PS_env_variables.png

Once the environment variable is set successfully, enter the command to list all pods.

kubectl get pods -A

mceclip20.png

You will get a message asking for the API Token which will be available from TMC. Click on your username -> My Account (under User Settings) to launch the 'My Account' page.

mceclip19.png

On this page, navigate to 'API Tokens' tab and click on 'GENERATE TOKEN' to generate a new token. Give it a name (for e.g. <your-username>-tanzu) and select the 'All Roles' checkbox. Click on 'GENERATE'.

mceclip17.png

Your token will be generated. Next, copy the token by highlighting it and using Ctrl + C (Windows) or Command (⌘) + C (macOS). Alternatively use your mouse right click button to Copy.

In the next step you will paste your API Token into the PowerShell CLI window. We also recommend that you save the API Token in a text file on your VMware Tanzu Horizon Desktop.


NOTE: Do not rely on the 'COPY' button to copy your API Token. This is a bug with this release of Tanzu. Please use the copy methods above to copy the API Token value. Do not exit this screen until you verify that you've saved your API Token.

mceclip18.png

Paste your copied token on your Windows PowerShell window by using Ctrl + V (Windows) or Command (⌘) + V (macOS) and hit 'Enter'. You may be asked to set the login-context name, set a name for it (for e.g. <your-username>-tanzu) and hit 'Enter'.

You will get a 'context successfully created' message along with a list of all pods running on your cluster. Now we're ready to configure CBC Container Security on this cluster.

mceclip21.png

To close the API Token popup screen click on the 'COPY' or 'PRINT' button to have the 'CONTINUE' button become clickable to exit this screen. 

To navigate back to the TMC Console screen, click on the App Launcher at the top right of the screen and from the drop down click on "VMware Tanzu Mission Control".

NOTE: This step is not required, but to show you how to get back to the Tanzu Mission Control Console in case you would like to explore it further or for debugging your cluster. 

Now let's go back to the PowerShell CLI window. 

navigate_back_to_TMC.png

To be able to run workloads on this cluster, we have to add a step to run the following kubectl command:

kubectl create clusterrolebinding tkgs-admin-privileged-binding --clusterrole=psp:vmware-system-privileged --group=system:authenticated

kubectl_clusterrolebinding_Step34a.png

 

2.2 Deploy CBC Container Security

On your newly deployed Tanzu Kubernetes Cluster, we will deploy CBC Container Security. To register your cluster manually, navigate to Inventory > Kubernetes > K8s Clusters > Add Cluster

mceclip24.png

Deployment consists of 4 steps:

1. Install Operator: Run the command on your PowerShell CLI

mceclip25.png

2. Configure Cluster Inputs:

  • Cluster Name: <username>-2 {Same name as what was created in Section 2.1}
  • Cluster Group: demo

mceclip26.png

3. Generate Secret:

  • Select existing key: TKGI
  • Click on 'Generate Secret'
  • Paste the command from Step 3 on your PowerShell CLI window and hit enter
  • Hit 'Next'

mceclip27.png

4. Finish Setup: Run the command given in this step on your PowerShell CLI and click Done when finished

mceclip28.png

Successful Installation will appear on the K8s Cluster page, refresh to reflect status changes from pending to running.

mceclip6.png


2.3 Configure CBC Container Security

Kubernetes default configurations, if not modified, can expose a high level of risk to an organizationMisconfigurations pose the highest risk to workload environmentsIn order to mitigate your organizational risk, security teams should start by enforce pod security standards. Pod Security Standards are defined by Kubernetesas well as governance bodies such as CIS Benchmarkswhich define what standard configurations should be modified in order to promote better security posture.

To start enforcing these Pod Security Standards teams will typically segment/organize their environment by building scopes I.e. Dev, Production, Application etc to ensure your like assets are protected in a uniform manner. For the purpose of this demonstration we will segment your clusters into your personal scope. 

Navigate to Inventory > Kubernetes > K8s Scopes

Select 'Add Scope' in top right to get started. 

mceclip29.png

A scope can be defined 3 different ways: cluster, namespace, or namespace in specific cluster. The more specificity defined the higher it will sit in the hierarchy for resolution.  

For this walkthrough select the focus of scope as Deploy Phase and add your cluster name in the Clusters selector by selecting it from the dropdown, then click Save.

mceclip2.png

For more details on scope creation navigate to Help > User Guide

mceclip30.png

Now that we have defined our demo scope, we will apply your preferred Pod Security Standard to this scope.

Navigate to Enforce > K8s Policies > Add Policy

mceclip31.png

Create a Policy Name and select your newly created Scope then hit Next.

mceclip9.png

Now you are viewing the Rules page, this page contains a variety of preconfigured rule sets which can control K8s configurations for: Workload Security, Network, Quota, RBAC, Volume, Command and CRDTo get started we will leverage the templated policies which contain the Pod Security Standard guidance. 

Select the 'Apply Templates' drop down.

mceclip10.png

CBC Container Security provides several templated policies to enable organizations to adopt baseline configuration standards as defined by Kubernetes and CIS Benchmarks. These templated policies allow our customers to enforce continuous compliance and enable governance and control over security posture of containerized applications. 

Templated Policy Breakdown:  

Basic Restrictive CIS Benchmark 1.6.0

K8 Baseline Pod Security Standard

  • Easy Adoption
  • Prevents Privilege Escalation
  • Non-Critical Applications
K8 Restrictive Pod Security Standard
  • Pod Hardening Best Practices
  • May inhibit some compatibility
  • Security Critical Applications or Low Trust Users

CIS Pod Security Guidance

  • Supports a Strong Security Posture
  • Focus CIS section 5.2.x 'Pod Security Policies'

Once you have chosen the desired template policy hit Next.

Typically when applying Pod Security Standards organizations run against challenges verifying the operational impact they will experience. CBC Container Security builds the ability to identify impact in the policy build pipeline. 

Review the built in violation simulation to get a picture of what workloads are violating these rules to-date and exclude or scope out where appropriate.

mceclip32.png

Once you have completed your review, click Save

mceclip11.png

 

Section 3: Compromising a Container


 

3.1 Generate Violation Alerts

Apply the compromised container yaml file:

kubectl apply -f https://raw.githubusercontent.com/octarinesec/security-demo/master/compromised-container/installation.yaml

mceclip14.png

Create a local proxy to reach the exposed dashboard service:

kubectl port-forward -n myapp service/dashboard 8080:8080

mceclip33.png

Visit the dashboard by opening http://localhost:8080 on your browser from within the VMware Tanzu Horizon Desktop

mceclip34.png

Hit Ctrl + C to exit out of the running proxy.

List the myapp pod. Copy the App name (Highlighted in screenshot below) by using Right Click or Ctrl + C

kubectl get pods -n myapp

mceclip16.png

Replace <dashboard-name> in the below command with the App name copied in previous step

kubectl exec -it <dashboard-name> -n myapp -c dashboard bash

mceclip17.png

List all running processes

ps aux

mceclip35.png

Finally, let's exit out of the dashboard using the 'exit' command

exit

mceclip0.png

 

3.2 Restricting Policy Violation Actions

Now that you have exec access, you are able to take the execution steps further to expose and compromise the application. For the purpose of understanding your risk and how CBC Container can help you mitigate that risk we will move from alert into a blocking posture.

Navigate to Enforce > K8 Policies select ‘edit’ on your configured Policy. 

Under Command, Modify “Exec to container” and “Port Forwarding” to Block.  

Hit Next > Next > Save

mceclip36.png

Now we will attempt to take the same actions on our second cluster protected via CBC Container Security Policy. 

Let us try to execute the local proxy command again to reach the exposed dashboard service. You will see that the request is denied by CBC Container Security "Blocked by Kubernetes security policies".

kubectl port-forward -n myapp service/dashboard 8080:8080

mceclip37.png

Once again, list all pods. Copy the App name (Highlighted in screenshot below) by using Right Click or Ctrl + C

kubectl get pods -n myapp

mceclip38.png

Attempt to Exec to dashboard app

kubectl exec -it <dashboard-name> -n myapp -c dashboard bash

You will see that the request is denied by CBC Container Security "Blocked by Kubernetes security policies"

mceclip39.png

On your local Browser, attempt to refresh the Dashboard once again. You will not be able to access the site anymore, verifying that CBC Container Security has restricted the vulnerable application.

Navigate to the CBC Console to Harden > K8s Violations to review the block notification verifying the enforcement of pod security standards.

mceclip40.png

 

This concludes the walkthroughPlease check out TechZone for more best practices and technical guides. Thank you for your time! Come back soon and try another walkthrough with us!

 

For Additional Support


Review Our Knowledge Base

Submit a Ticket

 

Have more questions? Submit a request

Article is closed for comments.