Overview

If you have been following this series then you will have seen the general overview and architecture laid out as well as the first stage of component deployments including Pivotal Ops Manager and BOSH Director for VMware. If not then make sure you check those out (and other articles in this series) using the links below.

• Part 1 Overview
• Part 2 Ops Manager and Bosh
• Part 3 NSX-T Prep
• Part 4 Installing PKS
• Part 5 Granting PKS Access
• Part 6 Cluster Operations
• Part 7 Cluster Access
• Part 8 Harbor Registry

In this article I am going to be looking at how we need to prep the NSX-T platform so that it can be used by PKS to deploy my Kubernetes clusters.

IP Schemes

If you are familiar with Kubernetes and its integration with NSX-T via the Network Container Plugin then you can probably guess what is coming next. I need to define the IP schemas I’m going to be using for the various parts of PKS, from Kubernetes down to Pods. These IP schemas are held as IP Blocks within NSX-T and sub-divided down as required for each Kubernetes platform that PKS deploys and for each POD that Kubernetes deploys to existing Kubernetes clusters.

In my PKS deployment I’m going to use the following schemas.

Use	Schema Type	CIDR	Gateway
PKS-K8S-Nodes	IP Block	175.10.0.0/16	N/A
PKS-K8S-Namespace	IP Block	175.11.0.0/16	N/A
PKS-K8S-LB-Services	IP Pool	172.25.1.0/24	172.25.1.1

For the Namespace and Pods IP Blocks the recommended subnet size is /16 as each PKS cluster created consumes a /24 network from the Nodes block and each namespace created in a Kubernetes cluster also consumes a /24 network. Using a /16 CIDR for the IP blocks ensures that the PKS environment can deploy 256 Kubernetes clusters and each cluster can create 256 namespaces (minus those namespaces that are created by default such as the default namespace).

The Services IP Pool is used for assigning out IPs to load balancer virtual servers that might sit in front of Kubernetes Pods that are deployed by end users (either separately as a “service”or part of a “deployment”).

There is a list of CIDR values that you should not use when creating your IP blocks as they are used internally to allow PKS to function. These are listed here for PKS 1.5 (current PKS version at the time of writing this article).

Creating the IP Blocks and Pools

Most historical documentation for PKS refers to creating all the necessary blocks and pools using the old NSX-T interface (“Advanced Networking & Security”). The latest version of our container plugin to orchestrate NSX-T (2.5.0 at the time of writing) does support the use of the policy interface so I’m going to take a chance and create all my blocks and pools using this interface instead. This is so I can see all my NSX config from the newer interface otherwise it will only be visible from the old interface. I’ll update this article if it proves not to work 😉

As my environment also supports native Kubernetes clusters which have their own address blocks and pools I have named all of the configuration for PKS use starting with “PKS-” to avoid confusion.

The important part to take note of for each of these objects is the unique ID. PKS requires the UUID for each rather than the friendly name. To fetch these I’m using the “Advanced Networking & Security” interface.

In the next article I’m going to be looking at deploying the PKS tile in Ops Manager to enable me to deploy and configure the PKS control VM. This will be a lengthy one as there are many steps! In preparation, get yourself a Linux VM ready that can reach your NSX Manager appliance from.