Building Enterprise-grade Kubernetes from scratch with Rancher Labs


Author: Ananda Kammampati

ananda at fieldday dot io >

Start Here

Following my journey in evaluating and building enterprise-grade Kubernetes platform from scratch on VMware ESXi hypervisor with different software stacks / frameworks, my next stop is to take a good look at what Rancher Labs has to offer.

The first thing that hits your face when you see Rancher UI is that it is SO CLEAN! Without knowing anything, you still feel curious and confident to navigate the UI. Simple but intuitive. It makes you feel at ease in understanding and getting familiar with, in a relatively shorter time.

Just like my previous articles on distributed Kubernetes and Photon-1.2, I am focussing of building distributed Kubernetes platform, but using Rancher Labs software stack this time. I am not venturing into the training aspect of Rancher Labs stack but I have provided links under 'Lab Exercises' for the same. Also, their YouTube channel is a great resource. 

What you see below is the Rancher UI showing a total of 9 virtual machines in total out of which, 3 VMs are dedicated for ETCD distributed key/value store, 3 VMs dedicated to host all ORCHESTRATION (Kubernetes) components and 3 VMs are dedicated for COMPUTE to run customer applications/workloads.



Having said that, there are 8 more virtual machines that work in tandem as part of this Hands-On lab. In total, I have 17 virtual machines that make up this single Hands-On lab. The approach is to build a truly distributed systems and have active HA wherever possible. So when I say " enterprise-grade " in the title, I truly mean it.

Needless to say, I am going to hand hold and walk you though every single component step-by-step (refer links under Sections), and showcase how to integrate them all and then finally install and configure Rancher Labs software stack. And at the end, you will have a fully distributed Kubernetes platform on top of it at your disposal. So hang tight!

Please note that I will be using terms VMs, hosts and nodes interchangeably.


Here is the Architecture diagram:


Here is the list of all the virtual machines, their roles and services they provide:


1 x EDGE VM running EFW distribution, configured as network gateway

1 x DNS VM - CentOS7 VM running FreeIPA for DNS service

1 x JUMPBOX VM - CentOS7 VM where you log in that has visibility to all the VMs. This is also referred to as the Bastion host.

1 x OPENFILER VM - to provide shared storage (NFS) for the backup setup ETCD nodes

1 x MYSQL VM - CentOS7 VM providing the Database service at the backend

1 x LB VM (Load Balancer) - CentOS7 VM running HAProxy server that is placed in front of two SERVER VMs

2 x SERVER VMs - CentOS7 VMs dedicated to run only the Containers that make up the Rancher Labs software stack

3 x ETCD VMs - CentOS7 VMs dedicated to run only the Containers that make up the distributed Key/Value Datastore

3 x KUBER VMs - CentOS7 VMs dedicated to run only the Containers that that make up the Kubernetes platform

3 x COMPUTE VMs - CentOS7 VMs dedicated to run only user Containers / workloads


The configurations of each VM is as follows:

Here is my physical setup:

  • I now have a powerful Intel Xeon based server with Supermicro board that which bought from MITXPC. Thanks to William Lam and Tai Ratcliff for their recommendations.
  • I physically connect my server to my Macbook-Pro laptop with a crossover cable back-to-back. And I define a dedicated network (10.1.1.X/24) between the two
  • I then use a second crossover cable to connect my laptop and my server's IPMI network interface for console access - for which I define a separate network (5.1.1.X/24)
  • I finally build enterprise-grade Kubernetes platform on my server with Rancher stack, and there by accessing it from my laptop

[ You may also want to check out our Hands-On-Box that we are designing specifically for our targeted audience ]

My MITXPC details with Web Client:


List of 17 virtual machines that make up Rancher-Kubernetes Hands-on lab.

In addition to the above 17 VMs, there is yet another virtual machine - REGISTRY that I tried using it as a private Registry with VMWare's Project Harbor. Since I didn't get a chance to play with it much this time, I am excluding it in this write-up. I am hoping to revisit private registry at a later time but that is something definitely worth considering as it is enterprise-grade.


So let's get started:

I will start with the pre-requisites first, followed by installing Rancher Labs software components on multiple nodes, followed by configuring them and then finally getting the distributed Kubernetes environment up and running.

I'll go in detail on the Pre-requisites as that is where I believe most of us go in circles, get frustrated and eventually give up. My sincere hope is that the documentations that I have posted here come handy as references in one way or the other, for your other projects as well.


Section #1: Pre-requisites

Section #2: Installing Rancher Labs software stack

Section #3: Configuring Rancher Labs stack

Section #4: Configuring Kubernetes environment

Section #5: Verifying distributed Kubernetes platform

Lab exercises:

Now that you have a fully built distributed Kubernetes platform at your disposal, the next logical step is to take some time in doing hands-on lab exercises for which, I would recommend the following:


a) Watch this webex recording  by Rancher Labs

b) Download the same YAML files shown in webex recording and practice the same exercises on your own lab. Redo the lab exercises, start tinkering the YAML files, till you get clarity and confidence

c) Join their community. I would certainly recommend signing up for their online meetups which are very informal and informative


For me, Rancher Labs feels like an IKEA thingie ( No offense 🙂 ) You don't need to be an expert but you will eventually figure things out and get it fully functional. If you are venturing into building distributed Kubernetes platform from scratch, Rancher Labs is definitely worth giving it a shot.

I sincerely hope you find this article informative and educational. Would love to hear your thoughts, suggestions and advice as I am planning to write more articles on this topic.

Wishing you all the very best on your plans with Kubernetes and Rancher Labs.

< ananda at fieldday dot io >