VMware vSphere withKubernetes 101An Introduction for vSphere Administrators
VMware vSphere with Kubernetes 101: An introduction for vSphere AdministratorsTable of ContentsIntroduction3Goals3What is Kubernetes?3What is vSphere with Kubernetes?4What components make up a Kubernetes Cluster?4How does vSphere with Kubernetes work?5Advantages for the VMware Administrator6What comes with vSphere with Kubernetes?6Supervisor Cluster . 6Tanzu Kubernetes Cluster . 6vSphere with Kubernetes & Cloud Foundation Services . 7VMware NSX . 7Tanzu Kubernetes Cluster or Supervisor Cluster: Which do I choose?8The Best Way to Get Started: VMware Cloud Foundation8Conclusion & Takeaways9Resources102
VMware vSphere with Kubernetes 101: An introduction for vSphere AdministratorsIntroductionIf you spend time in the information technology world you’ve likely heard the word “Kubernetes,” often in conjunction with containersand developers. Containers first started on Linux in 2008 and are a lightweight & portable way to distribute and run applications acrossoperating systems and clouds. Containers are not virtual machines, and by being lightweight they don’t have the same well-definedboundaries — security, performance, or even political — that virtual machines have. As you might expect, this has both challenges andadvantages.Containers can be incredibly useful for developing applications. Kubernetes was created to help manage many of the challenges arounddeploying those applications, most notably by helping automate and orchestrate deployments and availability.Kubernetes itself is an open-source project, governed by the Cloud Native Computing Foundation. VMware contributes heavily to theopen-source Kubernetes software base and is deeply involved in Kubernetes communities and governance.Kubernetes is extremely API-driven, which lends itself well to automation. It is very appealing to application developers as they seek toimplement modern development practices, with short or continuous development cycles, well-defined APIs, and clearly separated anddefined services which are often referred to as microservices.vSphere and Virtual Infrastructure administrators often find themselves positioned between developers seeking to implement modernapplication development practices and more traditional IT infrastructure and governance rooted in decades of practice. This guide isintended to help admins understand what vSphere with Kubernetes is, how it helps build bridges, and how to get started with this newand exciting form of infrastructure for modern, cloud-native applications both on-premises and in public clouds.GoalsAt the end of this document it is our goal that you will understand: What VMware vSphere with Kubernetes is The value that a Kubernetes Namespace brings to both the VMware Administrator and developers The differences between a Supervisor Cluster and a Tanzu Kubernetes Cluster How to get started with vSphere with Kubernetes & VMware Cloud Foundation ServicesWhat is Kubernetes?According to Kubernetes.io, Kubernetes is “a portable, extensible, open-source platform for managing containerized workloads andservices, which facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services,support, and tools are widely available.”What does that mean to a VMware Administrator? Kubernetes is an innovative approach for orchestrating the deployment and ongoinglifecycle management of modern, container-based workloads. Perhaps a brief history of the different approaches to applicationdeployment will help aid our understanding of how Kubernetes fits in the modern enterprise: Traditional DeploymentApplications and workloads deployed directly to physical servers are considered “traditional” deployments. Deployments of thesetypes tended to be inflexible, hard to scale, and wasted costly resources by trapping them on specific systems. Virtualized DeploymentVMware ESXi, a hypervisor, adds a layer of abstraction that allows for the creation of “virtual machines” which mimic the functions ofa standardized physical server such that a workload does not know it is not running directly on a physical server. Each virtual machinehas its own set of allocated resources as well as an operating system and can provide isolation of resources from other virtual machines.VMware ESXi also provides numerous availability features like vMotion, Dynamic Resource Scheduling, High Availability, and more, allof which provide massive advantages over traditional workload deployments. Containerized DeploymentContainers are like VMs but are lightweight and do not have the rigid boundaries that VMs have. This makes them more portable andagile within a family of guest operating systems (such as Linux). A container’s operating system comes from the system the containeris running on and is shared among all containers running on a host. However, containers have their own filesystems and resource3
VMware vSphere with Kubernetes 101: An introduction for vSphere Administratorsallocation mechanisms. Containers are popular because of the ability to have continuous development and integration for deployment,a capability made possible by their lightweight nature.What is vSphere with Kubernetes?VMware vSphere with Kubernetes was announced at VMworld 2019 as Project Pacific. It adds Kubernetes capabilities to vSphere in waysthat respect the traditional experiences of both developers and vSphere Admins.To a developer, vSphere with Kubernetes looks and acts like a standard Kubernetes cluster. Their tools and processes work acrossimplementations. They can use the Kubernetes “declarative syntax” to define what resources they need, such as storage, networking,and even relationships & availability requirements. By using the industry-standard Kubernetes syntax they don’t need direct access to,or knowledge of, the vSphere APIs, clients, or infrastructure.To a vSphere Admin, vSphere continues operating just as it has for decades but now with new workload management features to bettermeet the needs of developers. Management of vSphere is still done through the vSphere Client, PowerCLI, and APIs, as it has beendone for years. vSphere Admins can deploy “namespaces” – the Kubernetes term for managing resources and policies – and managethe security, resource consumption, and networking capabilities available to the developers.vSphere with Kubernetes provides a unified approach to infrastructure that is uniquely suited for hosting both traditional workloads, andmodern, cloud-native applications. For application developers, it is Kubernetes. For vSphere administrators, it is vSphere. For thebusiness, it is a consistent, standardized approach for deploying and managing traditional workloads alongside modern, cloud-nativeapplications, while safeguarding the security, compliance, and control of the IT infrastructure.What components make up a Kubernetes Cluster?There are many components that are part of a Kubernetes Cluster. Here is an explanation of the ones relevant to deploying andconfiguring vSphere with Kubernetes: NodesThere are two main node types in Kubernetes, a Master and Worker. A master node is a management node, what you would expectof vCenter Server. A worker node is what you would expect of an ESXi host, allowing you to run Pods. PodA Pod is a group of one or more containers. If we map this to a VMware Administrator construct think of Pods as an object similar to avirtual machine. Pods are managed by the Kubelet that runs on each node. Kubelet watches Podspecs assigned to it and handles alllifecycle by comparing actual Pod state to the desired state stored in the Podspec.4
VMware vSphere with Kubernetes 101: An introduction for vSphere Administrators StorageThe files stored within a container are ephemeral, which means on each container restart the data is lost. This is both an advantageand disadvantage. If you wish to have data be persistent it must be stored in a persistent volume. There are many different types ofvolumes available to Kubernetes. VMware vSAN has native container storage capabilities, allowing workloads to mount persistentvolumes inside the VMware Cloud Foundation deployment. vSphere Cloud Native Storage provides the capability to back Kubernetespersistent volumes with vSphere volumes. The CNS provider supports VSAN and any other VMFS based datastore. NamespaceA Namespace is used as the unit of management in environments with many users across multiple teams or projects. Namespaces area way to divide cluster resources and separate permissions between users. When a Namespace is created you assign CPU, Memoryand Storage limits to restrict the amount of resources a workload can consume, not unlike a vSphere Resource Pool. WhereNamespaces differ from Resource Pools is that they also incorporate controls such as security. For example, from a security perspectivevia Namespaces you can manage access controls by using edit or read-only groups. You also have the ability through security policiesto limit ports, audit changes and force encryption of data. To encrypt all containers and/or VMs in a Namespace is done by setting oneproperty rather than going to each VM and encrypting individually.How does vSphere with Kubernetes work?vSphere with Kubernetes introduces Kubernetes APIs as a new Developer API that provides a cloud service consumption experienceanalogous to what they would get in a public cloud while providing a new control plane, or management interface, for vSphere via theextended Namespace construct. This allows for deep orchestration and governance of workloads, whether they are containers,applications, or even virtual machines.vSphere with Kubernetes embeds the Kubernetes API, together with a custom management agent called a Spherelet, directly into theESXi hypervisor. The Spherelet is based on the Kubernetes “Kubelet” and enables the ESXi hypervisor to act as a native Kubernetesnode which can participate in a Kubernetes cluster. With this, every ESXi host can host containers directly on the hypervisor without therequirement for a separate Linux operating system (OS) instance. To accomplish this, we have added a new container runtime to ESXicalled the CRX. This is presented to Kubernetes as an ESXi vSphere Pod Service.The vSphere Pod includes a purpose-built and lightweight Linux kernel that is responsible for running containers inside the guest. Sincethis Linux kernel is provided by the hypervisor, VMware has been able to make numerous optimizations to para-virtualize the container,5
VMware vSphere with Kubernetes 101: An introduction for vSphere Administratorsboosting its performance and efficiency. Additionally, because the CRX kernel does not load a full Linux guest OS, the instantiation ofnew pods is very fast.Along with embedding Kubernetes directly into the hypervisor, the vSphere Client has also been made Kubernetes aware. Using thetraditional vSphere Client, we can now view and manage Kubernetes objects alongside our virtual machines. Conversely, Kubernetescan also specify and control some aspects of traditional virtual machines, helping to seamlessly blend traditional and container workloadstogether to form a cohesively managed application.Under the covers and invisible to the developers who consume Kubernetes clusters, vSphere with Kubernetes abstracts storage,network, and other resources. The developer can deploy containers or virtual machines without having to know or use traditional vSphereAPIs. They use Kubernetes the way they always have.Advantages for the VMware AdministratorFor the VMware Administrator, the introduction of Kubernetes as a control plane for vSphere opens possibilities for new workloadmanagement and orchestration in the future while still protecting your investments & efforts today. vSphere traditionally has been aboutmanagement of virtual machines and infrastructure while being somewhat indifferent to the actual applications running on the VMs.With vSphere with Kubernetes both the developer and the VMware Administrator can now easily create workloads and policies thatgovern containers, VMs, or both simultaneously. All aspects of application workload management are now first-class citizens in a vSphereenvironment.Developers may already be running container workloads in your environment, but as a VMware Administrator you have no awarenessnor visibility into them, making governance and troubleshooting difficult. With vSphere for Kubernetes, administrators gain visibility intoKubernetes workloads running on their virtual infrastructure. Enabling vSphere with Kubernetes allows you, as a VMware Administrator,to provide the platform based on the same performance, security, and availability criteria you use today for your traditional virtualmachine workloads. Developers will still be able to use their same tools to code, test, deploy, and support their applications. This givesyou as a VMware Administrator the ability to apply existing governance processes & tools to the environment, while developers haveaccess to the modern application self-service components they enjoy.What comes with vSphere with Kubernetes?There are many ways to deploy Kubernetes. Options including managed, cloud, on-premises virtual, and on-premises bare metal. Thereare tools such as the open source Minikube that have been developed to install and operate a Kubernetes cluster on a single host, whichis great for training. For enterprise use, though, most deployments require extensive setup work, new processes, and retraining of staffto install and operate Kubernetes effectively. This is where vSphere with Kubernetes and the VMware Cloud Foundation Services shine,with ease of installation and operation that fits naturally into your existing IT infrastructure and processes.Within vSphere there are two types of Kubernetes clusters that run natively: a “Supervisor” Kubernetes cluster control plane forvSphere, and the Tanzu Kubernetes Cluster, sometimes also referred to as a “Guest Cluster.”Supervisor ClusterThe supervisor is a special kind of Kubernetes cluster that uses ESXi as its worker nodes instead of Linux. This is achieved by integratingthe worker agents, Spherelets, directly into the ESXi hypervisor. The Spherelet doesn’t run in a VM, it runs directly on ESXi via vSpherePods. The supervisor cluster is a Kubernetes
Kubernetes itself is an open-source project, governed by the Cloud Native Computing Foundation. VMware contributes heavily to the open-source Kubernetes software base and is deeply involved in Kubernetes communities and governance. Kubernetes is extremely API-driven, which lends itself well to automation.