If you are new on this site, I can tell you that this is my first blogpost on this site. I hope you will like what you read and hopefully the content and information will get better as I will be posting blogposts regularly. If you want to read about me, who am I, my motivation and goals for this blog etc. I suggest you read my about page.
Alright then, this post will focus on some of the things you should consider before spinning up a home lab. If you, like me, dont have access to free hardware, this post will hopefully help you to have the right mindset when choosing the components for your specific usecase and potentially save you $$$ in bad decisions/hardware investments.
Disclaimer: Hardware not listed on the official VMware HCL is not supported by VMware.
Over the years I have seen an increased interest in small home labs, running VMware vSphere, for testing, educational purposes and for product reviews. Then a thought came to me – why not blog about this as I have invested a lot of time and $$$ for my home lab. My definition of home lab is based on consumer components (no rackservers, SAN etc). However the trending for home labs is that the requirements are growing and you might need to look into server components for meeting your lab requirements. When creating a home lab you need to consider a lot of things. I have applied parts of a design methodology for this:
What are your intentions with this home lab? I often see that people havent thought through on what they really want to use it for. I suggest you think about what your usecase is now and will it change in the near future? What requirements do you have for your usecase? Think of a design that will be easily repeatable and scalable if you need to add more compute, storage or network in to your lab.
Now that we know what you want to use your home lab for, we can start thinking of sizing. Lets say that you are taking the VCP exam. For that you probably need a vcenter server and a couple of ESXi hosts. This could fit on a single box. If you on the other hand need to implement the whole shebang like: vSphere, vCAC, vCO, vCops, nested ESXi’s etc, then you probably would need more boxes.
When you create your home lab, dont expect to see real enterprise performance. F.ex. if you try running 5 VM’s on a spinning SATA disk it will probably drive you nuts, and will kill you slowly. You need to think about what kind of performance will be acceptable for you, and choose the components that satisfies your performance requirements.
This is the fun part, and also where it can go bad if you dont do some research before investing in components. I have broken this down as there are many elements.
- Form Factor
Do you need to run expansion cards, attaching more than 2 disks, multiple processors etc. then you should consider buying cabinets (ATX)
If you dont need this (or like me – I am done with the chapter in my life where I would assemble the whole thing from scratch), then you can consider ie. Intel NUC’s, Mac Mini’s, or even powerful laptops.
Well this is an interesting spot as there isn’t any whitelist of working motherboards for ESXi – Yet (I am considering to create a list in the future). You really need to do some research which motherboards will work. You should look at whether the board supports Intel VT-x & VT-d, max memory supported, onboard controllers, Vendor of Onboard NIC (Realtek f.ex. require additional drivers), NIC count etc.
Is speed important in a home lab? Probably not. However what does matter is cores. If you need to run a lot of virtual machines you should go for higher cores. Your CPU must also support Intel VT-x/AMD-V and Intel EPT/AMD RVI if you are going the nested virtualization path and run 32 and 64 bit nested VM’s. Most newer CPU’s has these features.
One thing that you just cant have enough of! Again is memoryspeed important in a home lab? I would suggest to fill up the DIMM slots with memory as you probably will need it all at some point. Lets do a quick math. Just to run one instance of ESXi 5.5 takes up 4 GB. a server with all vCenter Server components on a single VM takes up 8 GB. You see where this is heading? At the end of the day your goal with the lab will tell you how much memory you need. But based on my experience, memory is always the ressource that first gets fully utilized.
Storage is always challenging. Free space is never the issue, IOPS and controller queue depth are. There are several possibilities here. You could run a NAS box, but with spinning disks, that could quickly become a bottleneck. The high end QNAPs comes with a SSD caching feature that do provide greater performance. Alternatively you could consider storage appliances, f.ex. Openfiler. Local SSD’s gives great performance, but the tradeoff is losing the benefits of shared storage. An alternative could be VMware Virtual SAN, but it does require 3 boxes – minimum. Are you thinking what is VMware Virtual SAN? Virtual SAN is a hypervisor converged storage solution from VMware that abstract and pools the local disks (min. 1 SSD and 1 spinning disk) on each hosts and provide you with one high performance shared datastore across your cluster. The configuration is really simple, just two chekboxes. I will be covering this in a future blogpost as I really like this product and is running in my home lab. I recommend installing ESXi on a USB stick – go for a min. 16 GB stick, this is also recommended by VMware. This will allow you to allocate space for the scratch partition on the USB stick. If you want to use Virtual SAN, you should not install ESXi on a disk as it will need to claim the complete disk.
Now that we have the other components in place, we need to think about our network. Do we need VLANs? Do we need multiple NICs? well this really depends on your goal. a single 1 Gbit NIC is sufficient to most home labs, but you may have different requirements. If you need to run workloads that requires multiple NICs you could consider running them in a nested environment. There you can spin up a ESXi host with max 10 NICs.
If you will have more than one box, then we need a switch to have them communicate with eachother. If you do not need to configure VLANs or Routing functionalities, then you could go with a unmanaged Gbit switch – but where is the fun in that? You could look into switches like the HP 1810 series. This only provides Layer 2 capabilities but with an affordable pricetag. If you need Layer 3 you could consider the Cisco SG300 which comes with static routing capabilities and the pricetag is in the same category as the HP switches.
- Form Factor
With these things in mind, you should be able to think of a design that will meet your requirements and save you for doing bad investments in hardware.
In my next couple of blogs I will cover the building of my home lab with subjects like:
- My goals
- My requirements, choices and justifications
- Components and Economy
- The building of my home lab
But home labs are not the only thing that will be on this blog… a lot of interesting stuff is coming.. follow me on this site, LinkedIn and on Twitter.